Kontakt large instrument loading freeze

[recallmenot]

Thanks for giving yabridge a shot!

• I read through both the troubleshooting and the known issues sections, and my issue wasn't listed there

Problem description

Kontakt opens normally.
When loading a big instrument (~1GB RAM or larger) in Kontakt, like audiobro LA scoring strings 3 or audio imperia chorus, after the small "loading" window disappears, Kontakt freezes for a good solid 3 minutes, freezing the DAW too.

What did you expect to happen?

On windows loading these instruments takes around 10 seconds.

What actually happened?

During the 2-3 minute freeze, the Kontakt yabridge process is pinned at 100% (indicating one core is active) but htop shows 4 cores active about 25% each.
The HDD that stores the library is barely active (HDD LED mostly off).

Then the DAW unfreezes, the instrument becomes fully usable and CPU usage goes down to usual levels.

Operating system

Manjaro latest

How did you install yabridge?

official repo

yabridge version

5.0.3

yabridgectl version

5.0.3

Wine version

both staging-tkg-8.1 and staging-8.1

Plugin

NI Kontakt

Plugin type

VST3

Plugin architecture

64-bit

Host

Reaper 6.69

Desktop environment or WM

Xfce

GPU model

12600k iGPU

GPU drivers and kernel

video-linux, Linux 6.1.11-1-MANJARO #1 SMP PREEMPT_DYNAMIC

Debug log

had to zip it (was 56.3MiB)
yabridge.log.zip

Anything else?

HDD that stores the libraries is:
NTFS as I use Kontakt on Windows too
mounted via fstab at boot
assigned the same drive-letter (winecfg) as in Windows so paths are identical

The libraries have been freshly batch-resaved in linux

[recallmenot]

ok just loaded the libraries into Kontakt in a Bitwig demo, load time is 5s now??? how??? so then something between yabridge and reaper appears to be broken.

had to compress it again, this time 19,3MiB
Bitwig_yabridge.log.zip

[dev-rfc]

I'm having the same issue with REAPER, I cannot open my projects since it takes forever to load

[dev-rfc]

Seems to be happening only with VST3 version of Kontakt

[robbert-vdh]

Looks like a bug in Kontakt, nothing I can do about it sadly. Looking at the log from the OP the plugin tells REAPER 99(!!!) times that its parameter names have changed (the IComponentHandler::restartComponent(16)). And each time REAPER does a full rescan of the plugin's 2607 parameters, which takes anywhere from one to four seconds in OP's log.

The only way to speed this up would be to fetch all parameter info at once instead of bridging the individual requests, but then you need to deal with cache invalidation and it goes against one of yabridge's main design goals.

[robbert-vdh]

@dev-rfc mentioned in the Discord that changing a compatibility option in REAPER helped. Do you happen to remember which options you changed? Maybe there's an option that causes the reloading of parameter values to be deferred until it is actually needed.

[recallmenot]

Hi,
so I did some more testing:
same Kontakt (vst3) on Windows with Reaper takes 5s to load these big Kontakt libraries.
I also tried a newer version of Kontakt (was 6.7.1, now is 7.1.6) on both linux and windows.
Windows+Reaper, same 5s. Linux+Reaper+yabridge, same 3 minutes.
Also tried updating Reaper to 6.75, same thing.
Here comes the disgusting thing: Reaper 6.75 in wine + Kontakt 7.1.6: 3s 🥇
Neither wine nor linux are slow, quite the opposite! It has to be something that yabridge does differently than the native implementation.

[robbert-vdh]

I already told you:

Looks like a bug in Kontakt, nothing I can do about it sadly. Looking at the log from the OP the plugin tells REAPER 99(!!!) times that its parameter names have changed (the IComponentHandler::restartComponent(16)). And each time REAPER does a full rescan of the plugin's 2607 parameters, which takes anywhere from one to four seconds in OP's log.

The only way to speed this up would be to fetch all parameter info at once instead of bridging the individual requests, but then you need to deal with cache invalidation and it goes against one of yabridge's main design goals.

Read the last sentence one more time. The only way to work around this buggy Kontakt behavior would be to fetch all parameter information at once and then deal with cache invalidation. That is possible, but I'd rather not do that to work around a buggy plugin.

[recallmenot]

Sorry didn't mean to offend you.
I was reporting my experiments in hope that with more information you or someone else here might have a different idea as to what it could be.
Also, I dug for the compatibility option you mentioned: Reaper usually runs plugins in the main thread. Setting Kontakt to run each instance in a dedicated process somehow seems to do a lot. The freeze now is down to around 10s which I can live with.
Might be worth adding this workaround to the FAQ on NI plugins.

But performance became quite bad, beginning to stutter with 2048 sample buffer size and only a few tracks.
I'm using Pipewire in Reaper by selecting ALSA, 0 in 2 out.
Maybe someone could also inform NI of the bug? I lack the technical expertise to even describe it to them :D

[robbert-vdh]

Maybe someone could also inform NI of the bug? I lack the technical expertise to even describe it to them :D

I chatted with an NI dev a couple days ago just to get an idea of if my hunch was right and it probably is. The NKS scripts included with instrument libraries are interpreted line by line, and if a script renames 100 parameters Kontakt isn't smart enough to wait a bit to batch those parameter changes. It will ask the host to do a full rescan of all of the plugin's parameter names 100 times. And NI probably won't consider fixing this a high priority because normally without bridging it only takes a couple seconds do those potential hundreds of rescans (though it's of course still a lot of unnecessary work regardless).

[recallmenot]

Thank you for all your effort!
I guess that wouldn't be something worth implementing a compatibility option in yabridge for, but I got two stupid ideas:
1.
If over 200 parameters got updated within 100ms, store the updates and when they stop, fetch the most recent ones to where they repeat and pass them to the daw.
Not sure that would make it any faster though...
2. Or maybe yabridge can feign deafness:
when it receives parameter updates, ignore and discard everything and when plugin stops updating, yabridge asks for the current version of all parameters and passes that on to the daw. Not sure how possible that even is.

[dev-rfc]

@dev-rfc mentioned in the Discord that changing a compatibility option in REAPER helped. Do you happen to remember which options you changed? Maybe there's an option that causes the reloading of parameter values to be deferred until it is actually needed.

Sorry for late reply, yeah I do remember what have improved in my case.

I couldn't properly load a project I previously saved because I had like 12 Kontakt instances and it was taking too long to load (around 30 minutes).

By running YABRIDGE_DEBUG_LEVEL=1+editor reaper I was able to see this message flooding the console lots and lots of times.

Then messing with some settings I changed the option "parameter automation notification" to "Ignore all notifications" and the problem is now gone. Still, I am unsure if that brings any side effects, but you can change this setting per plugin, so change it only for Kontakt if you need it.

Still Kontakt is quite slow on Wine compared to Windows, but at least it's at the same speed comparing it inside FL Studio (Wine) without bridging, so it's not a Yabridge issue it seems, especially because it seems to be happening in this particular scenario of REAPER(Linux) + Kontakt, none of my other plugins have caused it so far.

[dev-rfc]

when it receives parameter updates, ignore and discard everything and when plugin stops updating, yabridge asks for the current version of all parameters and passes that on to the daw. Not sure how possible that even is.

@recallmenot that's up to the DAW hosting the plugin to decides, and in that case, that option in REAPER does this trick.

[recallmenot]

In the plugin search window, you can set these compatibility-options per plugin, but for me that specific option does nothing. But banishing Kontakt to a dedicated process seems to work, no idea why.
Loading time still isn't as quick as in Reaper on wine though.

[TomRobinsonIO]

I seem to be having a similar issue where loading a large instrument (a drum library in my case) loads fine after a minute or two. But, trying to reopen a session with this instrument loaded causes the session to take around 30 minutes to start. Ignoring notifications doesn't seem to change anything and running Kontakt as a separate process causes [1] 89842 segmentation fault (core dumped) reaper when attempting to reload the project.

[robbert-vdh]

I implemented the parameter info batching for VST3 plugins in 8289d76. I'll do the same thing for CLAP soon. Can anyone who experienced this issue test if this makes a big enough difference? You can download a development build of yabridge here:

https://github.com/robbert-vdh/yabridge#installing-a-development-build

[TomRobinsonIO]

Hey @robbert-vdh, thanks for implementing the batching. I just ran through a couple of quick tests with a few projects and it appears to cut the load time down to around 10-15 seconds for smaller projects for me. I can do some additional testing later if need be. This appears to have resolved the issue though. Please let me know if there's anything else you'd like me to do. Again, greatly appreciate the work you've put into this.

[robbert-vdh]

The potential performance overhead of this VST3 Kontakt bug should be reduced by a lot in yabridge 5.0.5.

[recallmenot]

@robbert-vdh I had no idea you continued working on this, thank you!!!!
I've now tried it with yabridge 5.0.5, ~10 seconds to load LASS3 GUI, that's windows speeds and now Kontakt on Linux is viable for me!!
Same with Imperia Nucleus, that's now fixed too from what I can tell!
It's exciting to have working Kontakt on linux, I can reboot much less!
Thank you!