mumble
mumble copied to clipboard
mumble-voip
HRTF support (OpenAL Soft Backend?)
I read on the Mumble wiki about HRTF and I wonder if the implementation of an OpenAL Soft backend has ever been considered. OpenAL Soft can process HRTF calculations internally. It's fairly easy to achieve HRTF processing with OAL Soft, all you need is an HRTF data set (OAL Soft ships with a default one), set the listener's Position, At and Up vector correctly (Mumble already has this data through Mumble Link), place the player voices correctly (Mumble receives that data from other clients) and turn HRTF processing on.
Benefits of OpenAL Soft:
- FOSS (LGPLv2)
- Cross platform (both in terms of OS and CPU architecture)
- internal calculation of HRTF
- can be requested by app or enforced by user for all OpenAL apps
- calculation is optimized: intrinsics for NEON, SSE and AVX instruction sets
- over 50 compatible HRTF data sets to find one which matches your head shape best
- internal calculation of Doppler Effect (if desired)
- OpenAL EFX to simulate reverbation of environment (if desired)
- example video: https://www.youtube.com/watch?v=30fTc5t5QNU&fmt=18 (EFX is the successor of EAX)
Links: Official site of OpenAL Soft: http://kcat.strangesoft.net/openal.html OpenAL Soft repo on GitHub: https://github.com/kcat/openal-soft Demonstration video of OpenAL Soft HRTF: https://www.youtube.com/watch?v=UtFatWeEvGY
I used OpenAL when I made my terminal Mumble client, and it worked great.
From the OpenAL soft project page:
ALSA, OSS, DirectSound, PulseAudio, MMDevAPI, CoreAudio, Solaris, QSA, SoundIO, OpenSL, WinMM, PortAudio, "Null" Output, and a .wav writer are currently implemented.
@Hiradur Were you thinking that OpenAL replace all of Mumble's current audio systems, or just have it as an additional one? It seems like it could replace everything, but someone who knows the audio system better could probably tell me why this is a bad idea.
@bontibon Very nice, I'll check it out. For now I only propose to use OpenAL Soft as an additional backend not replacing the existing ones. I'm unfamiliar with how OpenAL handles input capturing and low level technical details so I can't say if it's suited to replace the existing backends or not.
In general, we are never opposed to new audio backends, as long as there is someone to maintain them.
We already have problems in this regard with PulseAudio/ALSA.
I agree, an optional OpenAL Soft backend would be really nice here for Mumble Link supported games in order to take advantage of the built-in HRTF processing.
Just adding some keywords here to make it easier to find now that the HRTF discussion in #2324 (simple binaural audio) has ended:
It would be impressive if mumble could do advanced positional audio with a head-related transfer function. The experience in Overwatch, where I could hear when someone was (for example) above and behind my left shoulder so reliably that I could swivel around to aim at them in a single motion, was excellent.
I'm pretty sure openal-soft could actually allow do ditch away all the OS-specific cruft, if one really wanted. And to the extent of my understanding, there really isn't any other cross-platform HRTF implementation (let alone open source)
On the other hand, from the point of view of an application, you actually simply just code against "OpenAL".. not openal-soft specifically.
https://ieeexplore.ieee.org/document/5661988 Did nobody know about this? @jfeldmaier
Yes, see https://github.com/mumble-voip/mumble/issues/2324#issuecomment-226331887.
I now uploaded the PDF file on our server: https://dl.mumble.info/docs/rothbucher2010.pdf
This looks interesting:
https://doc.qt.io/qt-6/qtspatialaudio-index.html
Qt Spatial Audio is an add-on module that provides a rich set of QML types and C++ classes to implement sound fields in 3D space. It contains an easy to use API for positing a listener in space, adding localized sound sources around the listener and emulating virtual rooms with reverb and reflections.