wwmm
Audio issue with Convolver when applications have different latencies
EasyEffects Version
8.0.5
What package are you using?
Flatpak (Flathub)
Distribution
Pop! 22.4
Describe the bug
While using the convolver as my only effect with an AutoEQ generated IR(this isn't that relevant because it also happens with wet set to -inf). I noticed that if I have something pausing or unpausing with a different latency it made the audio go loud like the IR was turned off. This happens even if the application is excluded.
But I noticed that if I changed the settings of the application that was pausing and unpausing to match the latency this doesn't happen.
This is also doesn't happen if the application that is pausing and unpausing has worse latency
Cases: Brave playing audio 21.3ms
Blender set to 16384 Samples(65.3 ms) (no issue) Blender set to 8192 Samples(22.7ms) (no issue) Blender set to 8192 Samples(10.7ms) (the issue happens) it also happens with sample rates below it
(Blender is set to use Pulse Audio, I am unpausing and pausing playback of a project, 96Khz Float 32(which matches the IR and device samplerate))
this issue seems to only happen with the convolver I have the same EQ setup as a 31 Band EQ using equalizer. In that case it doesn't happen.
It also happens between different applications Brave and Blender were just example
Expected Behavior
For it to not turn off the convolution.
Debug Log
Debug Log
Paste your log here
Additional Information
No response
When pipewire changes latency the audio buffer size changes and the convolver has to go through a full reset of its internal state. The reason is that as far as I know the zita-convolver library we use needs to be reconfigured when the audio buffer size changes. Maybe I am wrong and there is a way to avoid that. I do not know.
The only way I can think of avoiding this problem would be to force a fixed array size inside the filter and make zita-convolver always do that. But that has the side effects of introducing extra latency whenever pipewire's audio buffer size becomes smaller than the filter internal buffer.
Another side effects of avoiding the reset is that old data would be kept inside the filter and played the next time the pipeline is active again. What seems to really bother some users. In short it is a hard problem to solve.
I don't really have much understanding of programming but I have some understanding of Audio Processing.
If I understand it right the current implementation changes the buffer size if something plays that has a lower buffer size/latency.
Option 1(fixed/highest latency leader): Make the audio with lower latency delay/buffer it's output so you don't need to change the buffer of convolver. This of course introduces latency. Maybe this could also be done where user select there fixed buffer rate like in many audio programs. With an option to keep old behavior maybe?
Option 2(mute): Mute the audio when the buffer resets. This way is crude but it's better than being blasted with +8db audio when the convolution EQ gets turned off.
Option 3(multi-instance): Multiple instances of Convolver. Convolver is pretty well optimized. When the buffer changes you could open another instance for the lower latency app. Impact could be further reduced by killing the lower latency instance. And it could possibly combined with the delaying method mentioned in the first option to make transition smoother. This of course spikes CPU usage when it happens.
I definitely think this needs addressing in some way because the current behavior feels like a bug. At least in my opinion.
Maybe this could also be done where user select there fixed buffer rate like in many audio programs.
It is not possible. PipeWire has full control over when plugins process or not something. As well as over which is the buffer size and sampling rate clients will have to work with. What the filters do is just registering in PipeWire a function that will receive the audio buffer. And at any time pipewire may change buffer size and sampling rate.
Mute the audio when the buffer resets. This way is crude but it's better than being blasted with +8db audio when the convolution EQ gets turned off.
That should be possible. But it will also cause a different kind of annoyance. In the cases where "blasting" the user is not a problem people will notice that suddenly there is no audio. And in a case where volume was already loud the system would alternate between silence and loud audio. What is also bad.
Multiple instances of Convolver.
That actually complicates the implementation by quite a lot. And it may still cause audible artifacts. As I explained above all the filter can do is registering a processing function and deal with whatever pipewire is sending every time the function is called. After the audio buffer changes keeping calling the extra instance is not viable anymore. So a hard switch is necessary. And will feel almost the same as destroying and recreating a single instance.
What I think that can be done is providing a configuration option that forces the convolver to use a fixed internal buffer size. And adding extra latency when necessary as I explained in my last post. Part of the code necessary for this is already there for different reasons. At least in the past there were hardware that made pipewire to work with buffers size that do not follow a power of 2. In order to deal with this possibility I wrote a code that makes the convolver switch to a fixed internal buffer when the size is not a power of two.
That being said just adding a switch is not enough. More code to make the transition between operation modes on the fly is necessary.
Mute the audio when the buffer resets. This way is crude but it's better than being blasted with +8db audio when the convolution EQ gets turned off.
I see now that if the convolver is not ready we put it in bypass mode. What means that any value to the input and output gain is ignored. Maybe this contributes to make things worse. I usually do not set input and output gains to it so I did not pay attention to that. Do you have a custom input or output gain set?
I've updated our master branch and the convolver will now play silence while it is still being initialized instead of an abrupt migration to the bypass mode. It is not the true solution but it is simple to do.