VST3/CLAP wrappers fail to call their internals' drop glue when destroyed
When removing/destroying an instance of a VST3/CLAP plugin made with nih-plug (using nih-plug-egui for the UI) inside a host DAW, the plugin's drop glue consistently fails to run. After some debugging, I was able to track this down to an incorrect Arc refcount in the wrapper code. This leaks an arbitrary amount of memory every time - in the case of my own sampler plugin, anywhere up to hundreds of megabytes.
Tested on FL Studio 2024 and Reaper v7.40 using VST3 and CLAP. Both print an equivalent nih_debug_assert_eq!() failure:
VST3: Debug assertion failed: Arc::strong_count(&self.inner) != 1 inside <Vst3Wrapper as Drop>::drop(&mut self)
CLAP: Debug assertion failed: Arc::strong_count(&this) != 1 inside unsafe extern "C" fn destroy(plugin: *const clap_plugin)
In the case of CLAP, I was able to check the refcount at the point of the assertion failure:
The problem still occurs if the plugin window is never opened.
Likely related:
In the VST3 wrapper, bg-thread fails to shutdown when the Plugin instance is destroyed, blocking on tasks_receiver.recv() forever and causing an access violation if the host unloads the plugin DLL (which FL Studio does when there are no more instances of the plugin in memory).
I was gonna submit a separate issue, but this definitely seems related the more I look into it. The CLAP wrapper stores the BackgroundThread handle separately from its plugin_data member, which means it gets dropped properly when the plugin instance is destroyed. The VST3 wrapper stores everything inside Arc<WrapperInner>, meaning none of it gets dropped properly and the thread never receives the "sender disconnected" signal from the EventLoop's background_thread member.
All the more reason to consider this a pretty critical bug, IMO.
That's interesting. I'm not sure how to check if it's the same problem, but I'm running a Windows 11 virtual machine on a Linux host and am using FL Studio inside it, and whenever I try to remove Spectral Compressor, Diopser or Crisp from the plugins list, or go to another project from one that has one of those, it freezes. After reading this issue I went to test something, and I just noticed right now, that it only happens whenever there's no more of the same plugin, just like you said. For example Spectral Compressor. I can remove as many instances of it as I want, but the moment I get rid of the last one, FL Studio freezes. I'm pretty sure this is the same problem
Not sure if this is new information but I thought I'd mention lol, I hope that PR gets merged soon