Complex convolution down/upmixing

[kode54]

I have produced a convolver singlet that may be useful for up and downmixing using complex impulse response presets. Included with the singlet is a trimming utility, and some impulse responses I generated at various sample rates, and a simple console tool for testing the convolver.

https://github.com/kode54/dh

I would like to see, at the very least, the ability to configure this driver to route a 6 channel signal, and for the application to support volume control for that, and downmixing to stereo headphones using this convolution code.

On my Retina 5K iMac with Core i7 4790K, using the vDSP method, I see speeds of over 110x real time, which equates to less than 1% of a single processor core.

Perhaps presets could be recorded or generated for complex upmixing of stereo to surround as well? The possibilities are endless.

[wildeyes]

Could you explain what's the feature you want added in layman terms?

[kode54]

With the code I've posted there, as demonstrated by the test application, it is possible to downmix 5.0 surround (the LFE impulse is silent) to a headphone or earphone mix. It would require the user mode driver to present a 5 or 6 channel source first, though, and it would require the application to support downmixing.

I only suggested the impulse convolver provided as a means to upmix stereo or 4.0 to 5.1 or even 7.1 discrete channels, if anyone feels like sampling an upmixing algorithm.

The objective is a free and open source competitor to Hajo's Headphone Enhancer, one that also happens to support per-application volume control, since it's not really possible to combine the two applications in a feasible way.

[kode54]

I figured out something else, that would involve another developer's filter constants, and Apple's vDSP framework for biquad IIR filters, which would work perfectly for this, and also be zero latency, since it could operate on any arbitrary amount of input samples fed into the driver, or however much you can output at once. It just needs to prepare 5 biquad filters, one each for the left speaker output of front-left, front-right, rear-left, rear-right, and center/LFE combined. Then two filter history states for each input channel (or pre-sum the center/lfe so we only have 5 pairs instead of 6).

This filter, the driver needs to operate at 48kHz, and there are up to 35 samples of delay. The symmetrical opposing front channel for front-right's response in the left ear, and left's response to the right ear, is delayed by 11 samples. The rear channels' response, the same-side has no delay, while the opposite side has a delay of 35 samples.

https://developer.apple.com/documentation/accelerate/1450374-vdsp_biquad_createsetup

This filter thing makes it possible to create as many instances as there are unique filter sets, and it will apply all of the stages of the filter (there are 16 stages per filter, applied one after the other)...

Hmm, maybe this is too complicated. Basically, I can work out the audio processing logic, such that it filters the input with zero input to output latency. I just need a version of the user mode driver that processes 6 channel (5.1) audio rather than stereo, or in addition to stereo. Then I need a version of the Background Music app that is aware of the channel maps of the audio it pulls from the user mode driver, and I can insert the downmixing algorithm into it.

I don't know that Apple's vDSP method is usable for this, since the IIR algorithm that the original filter uses, feeds identical input from the input channels into every single stage of the IIR filters, and sums all of their outputs into the final output, along with a single passthrough level of the original input samples, rather than running each of the filters in parallel to each other. Sort of like a 16 band equalizer.

[kode54]

Since this hasn't gone anywhere in 5 years, here's something that may help, since I don't think I can do it either:

https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Simple-audio-post-processing

This documents how to use jconvolver to do what I was describing, which in turn uses FFTW to do convolution and mixing according to their plaintext configuration file. Of course, it's limited to JACK API.

[kode54]

Here, take a look at this filter I made:

https://github.com/losnoco/Cog/blob/main/Audio/Chain/HeadphoneFilter.m https://github.com/losnoco/Cog/blob/main/Audio/Chain/HeadphoneFilter.h

It only needs adapting to replace how it loads the impulses, and maybe replacing RetroArch's resampler with any of your own choosing, accounting for input/output latency at the start/end of the samples.