Suggestion on How To Implement Near Field filtering on Generic Far Field HRTF

[ABoredBunny]

I wrote beforehand about the want for a generic near field filter for headphones. And I was made aware of the fact that the ambisonic mode already has multi field HRTF.

But Multifield SOFA files are hard to come by and EAC Individualized HRTF Synthesis makes it very easy to generate personalized HRTF, but they aren't multifield either.

Then I noticed one of the VST plugins I use has Near Field HRTF with generic HRTF SPARTA Binauriliser_NF.

And found out they implemented it based on this paper which uses filtering based on direction and distance, based on a table.

Here's a version of it in Matlabs.

You could use it in different ways. Like using it to estimate near field in "full" mode.

Or implementing it as an extra option in the MakeMHR commandline, to upscale HRTFs.

[ThreeDeeJay]

You could open a feature request on EAC to add multi-field HRTF generation, but yea it'd be nice to implement it directly into OpenAL Soft's MHR generator for SOFA files without multi-field/NFC as well. 🤔

[ABoredBunny]

Apparently EAC already has a function to interpolate stuff. Possibly even including distance. With several algorithms, including "state of the art" spherical interpolation. But it isn't in the easy to use app. You need to use matlabs itself. Do you have any pointers how to run matlabs files? @ThreeDeeJay

[ThreeDeeJay]

@ABoredBunny It's a lot easier now that the app was released as a "standalone" (it doesn't require the MATLAB app, just the runtime, which is a few GBs but otherwise it's now pretty straightforward.)

[ABoredBunny]

@ThreeDeeJay I know about that. I mean that the matlab based version(not the app) has an option to interpolate distance and points. But I don't know how to use the MATLAB "scripts". This should also solve your issue and create a useable HRTF that works in makemhr, if we get it running.

[ThreeDeeJay]

Ohh, I see. I was already planning to request generating HRTFs with uniform elevations so I'm glad to hear there's already a function that should hopefully make it easier to implement 👍

[SDX-LV]

Hi, if you need to generate HRTF with multiple depths, Mesh2HRTF can do it simply by specifying "PlaneHorizontal" as one of the evaluation grids. You can see what this grid looks like here: https://github.com/Any2HRTF/Mesh2HRTF/blob/develop/mesh2hrtf/Mesh2Input/EvaluationGrids/Data/PlaneHorizontal/EvaluationGrid.pdf

Note this specific grid has way-too-many points, but it is not necessarily a bad thing. And it is not too complex to create any other evaluation grid that includes both more-distant and fewer nearfield points.

No guarantees if the files SOFA produced by Mesh2HRTF will be correctly loaded by any software that can use depth information.

[ThreeDeeJay]

@SDX-LV By the way, unless Mesh2HRTF plans to support exporting multi-field SOFA, apparently there's a way to merge multiple SOFAs with differen depths into a single MHR by using a .DEF file as explained in https://github.com/kcat/openal-soft/issues/722#issuecomment-1182381899 So it might be worth a shot.

So if I understood the SCUT_KEMAR.def Right. rate = 44100 points = 512 radius = 0.09 distance = 0.2, 0.25, 0.3 azimuths = 1, 24, 36, 72, 72, 72, 72, 72, 72, 72, 36, 24, 1; 1, 24, 36, 72, 72, 72, 72, 72, 72, 72, 36, 24, 1; 1, 24, 36, 72, 72, 72, 72, 72, 72, 72, 36, 24, 1 [ * ] = sofa : "./SCUT_KEMAR_radius_0.2.sofa" [ * ] = sofa : "./SCUT_KEMAR_radius_0.25.sofa" [ * ] = sofa : "./SCUT_KEMAR_radius_0.3.sofa" This is how it would look like, if I tried to make an HRTF using multiple sofa files(using unpacked SCUT_KEMAR)? Or do I need to do something special so it understands what to do with multiple files? - ABoredBunny

Possibly. Sofa support is a rather late addition for makemhr, added by someone else, and I'm not quite sure how the [ * ] for sofa files will act with multiple sofa files, but a quick look at the code initially appears that would be correct. - kcat

On a side note, I converted SCUT_KEMAR_radius_all.sofa to MHR to try NFC as suggested here, and it seems to work, although to be honest not quite as convincing/pronounced (perhaps nfc-scale needs adjusting?) as CMSS-3D + MacroFX but still a lot better than a constant distance:

https://user-images.githubusercontent.com/71472458/196340196-b799f5c0-ee06-4c54-adf4-7b108d10edf2.mp4

[ABoredBunny]

@SDX-LV I know Mesh2HRTF is better, and produces "Academicaly accurate" HRTFs with arbitrary sample points(Because I'm pretty sure it works by calculating a volumetric pressure map). But Mesh2HRTF requires a 3d model of a persons head. It is also very UNIX oriented(I'm pretty sure you're the guy that made the windows tutorial for it, but I did face issues when I tried it)

It will also take several hours to compute on most average computers.

While EAC is a much more approachable technolegy. Only requiring a few measurements using a caliper, or even just a photo with reference object for most measurement. Computing in about a minute on my 9 year old midrange CPU. And has an app that auto installs MATLAB and the programm.

@ThreeDeeJay You don't even need multiple fields in Mesh2HRTF, you can create a sample grid of your choice, and have it all in one.

In terms of Near field effect, you could play with the distance scale. But the OpenAL soft multi field method is more accurate. MacroFX uses approximation based on an algorithm, because it was made when Multi field HRTF with blending in between(What I think Openal soft uses) would have been pretty memory intensive. Here's the MacroFX White Paper

Also you probably forgot to set the hrtf-mode to "ambi3" Because it doesn't work in "Full" mode.

[ThreeDeeJay]

@ABoredBunny Hmm this is weird, I get the opposite results: I tried setting hrtf-mode=ambi3 but then I couldn't hear the near field effect anymore.

https://user-images.githubusercontent.com/71472458/196626301-e1f06da3-de3b-4790-bb50-8b63a3a7b949.mp4

Also tried nfc=true but it didn't help (I think that one's for surround tho) and I'm not sure what value I should use for nfc-scale in case it's wrong (which I doubt since it seemed fine with CMSS-3D, unless DSOAL/OpenAL Soft use a different scale) It also switched to ambi2 here (I think it was 1.22.2), so I updated to the latest build from GitHub actions and it kept using ambi3, but the near field effect was still gone. 🤔

[kcat]

On a side note, I converted SCUT_KEMAR_radius_all.sofa to MHR to try NFC as suggested https://github.com/kcat/openal-soft/issues/722#issuecomment-1179484470, and it seems to work, although to be honest not quite as convincing/pronounced (perhaps nfc-scale needs adjusting?) as CMSS-3D + MacroFX but still a lot better than a constant distance:

It's possible the way makemhr tries to normalize the HRTF volume for each field/distance causes the near-field effect to become dulled, by not allowing it to become as loud when nearby as it otherwise would. This normalization is needed to remove any distance attenuation there may be on the more distant responses vs the closer ones (wouldn't want the 0.5m responses being +6dB louder than the 1m responses, ignoring the near-field effect), so if that's what's causing it, I'd need to find a better way to keep the near-field effect as intended without causing the distant responses to be quieter than desirable.

It's also possible CMSS-3D + MacroFX applies a boost to make the effect more pronounced, for a stronger "wow it's right next to me" sensation, whereas those HRTFs model it more realistically.

Also tried nfc=true but it didn't help (I think that one's for surround tho)

nfc=true is needed for applying the NFC filters with HRTF. I'd actually consider it a bug that full HRTF still uses it without it being enabled since the effect may not play nicely with a game's unit scale. As for it not seeming to work even with nfc=true, I'd need to see it along with a trace log. It's working here, but it does need to be fairly close to really feel it.

[ABoredBunny]

@ThreeDeeJay Maybe It's your sound card, it seems to create 6 channels, maybe try setting it to stereo in windows. Here's how it sounds for me:

https://user-images.githubusercontent.com/65722212/196693445-b6d1a456-2a15-4a4e-b843-cd1e178cde4c.mp4

Edit: Might also be a different Compilation flag in makemhr, I for example disable the diffuse field EQ, because I use a seperate EQ SCUT_48000.zip

[ABoredBunny]

@ThreeDeeJay Do you have a compiled version of the new Makemhr? Now that EAC has a new version that allows interpolation, it should be possible to use those HRTF's. I don't really have the "Enviroment" setup for that stuff. So it would be helpful.

[ThreeDeeJay]

It's possible the way makemhr tries to normalize the HRTF volume for each field/distance causes the near-field effect to become dulled, by not allowing it to become as loud when nearby as it otherwise would. This normalization is needed to remove any distance attenuation there may be on the more distant responses vs the closer ones (wouldn't want the 0.5m responses being +6dB louder than the 1m responses, ignoring the near-field effect), so if that's what's causing it, I'd need to find a better way to keep the near-field effect as intended without causing the distant responses to be quieter than desirable.

It's also possible CMSS-3D + MacroFX applies a boost to make the effect more pronounced, for a stronger "wow it's right next to me" sensation, whereas those HRTFs model it more realistically.

I see. I'll try to get something like this working to test the SOFA directly to see how NFC compares before and after converting to MHR. 🤔

nfc=true is needed for applying the NFC filters with HRTF. I'd actually consider it a bug that full HRTF still uses it without it being enabled since the effect may not play nicely with a game's unit scale. As for it not seeming to work even with nfc=true, I'd need to see it along with a trace log. It's working here, but it does need to be fairly close to really feel it.

@kcat Here's a zip file containing RightMark 3D Sound pre-configured with NFC (SCUT_KEMAR_radius_all), dsoal-aldrv.dll https://github.com/kcat/openal-soft/commit/02d0b3495a79a4e8b0070dae6e8e4deed87fed6c, dsound.dll https://github.com/kcat/dsoal/commit/910cf95edc7c36113e949fa073cae637d3b44557, the running water sound, channels=stereo, stereo-mode=headphones, hrtf=true, hrtf-mode=full, nfc=false and the respective logs: RightMark3DSound_NFCTest.zip

@ThreeDeeJay Maybe It's your sound card, it seems to create 6 channels, maybe try setting it to stereo in windows. Here's how it sounds for me: Edit: Might also be a different Compilation flag in makemhr, I for example disable the diffuse field EQ, because I use a seperate EQ SCUT_48000.zip

This time I made sure the speaker configuration was set to Stereo in the Windows sound settings, but I still got Allocating 6 channels in the log, tho it doesn't seem it made any difference. And I'm not quite sure I hear the nfc effect in your demo. To me, it just sounds like it gets bassier the closer it gets, which I think happened to me with a certain OpenAL Soft build, but doesn't seem to happen with the latest one that I attached above.

@ThreeDeeJay Do you have a compiled version of the new Makemhr? Now that EAC has a new version that allows interpolation, it should be possible to use those HRTF's. I don't really have the "Enviroment" setup for that stuff. So it would be helpful.

I managed to set up Github actions to compile the latest makemhr.exe, but I still wasn't able to convert EAC SOFAs to MHR, even with the recent custom grid (uniform layout/elevations) feature. I'm planning to open an issue so let me know if you get the same error.

Generating "EAC_Default_CustomGrid_-90_90_10_0_360_10_48kHz-48000.mhr"
Using 2 threads.
Reading HRTF data from EAC_Default_CustomGrid_-90_90_10_0_360_10_48kHz.sofa...
Unexpected delay attribute: _Netcdf4Coordinates = <null>
Unexpected IR attribute: _Netcdf4Coordinates = <null>
Detecting compatible layout...
Non-singular poles on field distance 1.000000.
Using 0 of 684 IRs.

[ABoredBunny]

@ThreeDeeJay

And I'm not quite sure I hear the nfc effect in your demo. To me, it just sounds like it gets bassier the closer it gets, which I think happened to me with a certain OpenAL Soft build, but doesn't seem to happen with the latest one that I attached above.

That is what the Near field effect is supposed to be, sounds at a certain distance start to get bassier(There is another problem regarding the near field, regarding parallex where sounds seem to come from the wrong direction because of things like head occlusion, but that is harder to fix, unless you have fancier algorithms or multiple fields) This increased bassiness is a distance clue for your brain.

I'm using the latest github actions build, because the newest release version doesn't have near field scale.

But I think I found your issue, the water sound has no bass. But the Near field effect is mainly about that.

I managed to set up Github actions to compile the latest makemhr.exe, but I still wasn't able to convert EAC SOFAs to MHR, even with the recent custom grid (uniform layout/elevations) feature. I'm planning to open an issue so let me know if you get the same error.

Thanks:

Using 4 threads.
Reading HRTF data from PERSONAL 2_48kHz.sofa...
Unexpected delay attribute: _Netcdf4Coordinates = <null>
Unexpected IR attribute: _Netcdf4Coordinates = <null>
Detecting compatible layout...
Non-singular poles on field distance 1.000000.
Using 0 of 2664 IRs.

Same Issue

I also get

Using 4 threads.
Reading HRTF data from PERSONAL 2_44.1kHz.sofa...
Error: Could not load PERSONAL 2_44.1kHz.sofa: Invalid format

When trying other sample rates like 44.1Khz and 96Khz

I hope it gets fixed.

[ThreeDeeJay]

That is what the Near field effect is supposed to be, sounds at a certain distance start to get bassier(There is another problem regarding the near field, regarding parallex where sounds seem to come from the wrong direction because of things like head occlusion, but that is harder to fix, unless you have fancier algorithms or multiple fields) This increased bassiness is a distance clue for your brain.

I'm using the latest github actions build, because the newest release version doesn't have near field scale.

But I think I found your issue, the water sound has no bass. But the Near field effect is mainly about that.

That was my first guess, too; but I tested with the original drum beat loop1.wav with RightMark3DSound_NFCTest.zip (which included the latest OpenAL Soft as of posting it) and I still hear the NFC effect (although subtle) without getting noticeably bassier whith proximity:

https://user-images.githubusercontent.com/71472458/198350509-5b6d8668-6439-475c-aab0-3fb71549ecd6.mp4

Same Issue

I also get

Using 4 threads.
Reading HRTF data from PERSONAL 2_44.1kHz.sofa...
Error: Could not load PERSONAL 2_44.1kHz.sofa: Invalid format

When trying other sample rates like 44.1Khz and 96Khz

I hope it gets fixed.

Yeah, I also got that error with another HRTF but I thought it was a fluke. At least it's not just me lol You might wanna report at least the first case/SOFA here as well: https://github.com/kcat/openal-soft/issues/779

[ABoredBunny]

@ThreeDeeJay What you're hearing is just the effect of sounds getting louder as it gets near. Which is serperate from the near field effect. Kind of seems like a bug, it is supposed to sound like in the video, where very close sounds get a bass boost.

The main difference in your alsoft and mine is "stereo-encoding" mine is set to hrtf, but that is unlikely to make a difference.

[kcat]

@kcat Here's a zip file containing RightMark 3D Sound pre-configured with NFC (SCUT_KEMAR_radius_all), dsoal-aldrv.dll 02d0b34, dsound.dll kcat/dsoal@910cf95, the running water sound, channels=stereo, stereo-mode=headphones, hrtf=true, hrtf-mode=full, nfc=false and the respective logs: RightMark3DSound_NFCTest.zip

If I use the drum loop instead of the water loop, I can hear a subtle bass boost when it gets near. It's not as strong as in @ABoredBunny's video, though. If I use hrtf-mode=ambi3 (and nfc=true) then it sounds more like it, though not exactly the same. Perhaps there's an issue with diffuse field equalization, since they said they turned that off.

This time I made sure the speaker configuration was set to Stereo in the Windows sound settings, but I still got Allocating 6 channels in the log, tho it doesn't seem it made any difference.

That's normal for full HRTF output. 4 channels for the main mixing buffer (first order B-Format) plus 2 channels for the actual device output:

[ALSOFT] (II) 1st order + Full HRTF rendering enabled, using "SCUT_KEMAR_radius_all-48000"
[ALSOFT] (II) Channel config, Main: 4, Real: 2
[ALSOFT] (II) Allocating 6 channels, 24576 bytes

I also get

Using 4 threads.
Reading HRTF data from PERSONAL 2_44.1kHz.sofa...
Error: Could not load PERSONAL 2_44.1kHz.sofa: Invalid format

When trying other sample rates like 44.1Khz and 96Khz

That looks like a libmysofa problem. mysofa_load returns NULL and sets the MYSOFA_INVALID_FORMAT error value. Best to report it there.

[ThreeDeeJay]

Ah, yes; I was able to replicate it with hrtf-mode=ambi3 and nfc=true but it still sounds off to me: not just the head-pounding bass but also how it has little to no effect on simulating close proximity like CMSS-3D MacroFX I mentioned before.

https://user-images.githubusercontent.com/71472458/198658102-42db3a08-0cdf-4057-aad0-fb74d343dc22.mp4

Anyhow, I know NFC is still a work in progress so I'm still optimistic about future improvements 👍 For future reference, here are other spatializers that do fairly convincing NFC without getting too bassy on both ears like that:

3D Tune-In Toolkit WARNING: Gets a bit too loud. way worse in the app, but it's one of the most natural-sounding NFC spatializers I've tested so far (excluding binaural recordings of course):

https://user-images.githubusercontent.com/71472458/198624373-8192f31a-09b7-4e8e-9f60-11c15552930b.mp4

Anaglyph VST

https://user-images.githubusercontent.com/71472458/198624284-65e9618a-8439-4d04-bc7d-44ad29c8ae72.mp4

Also now that you mention it, attempting to convert the EAC SOFA to HeSuVi HRIR via ffmpeg+sofalizer(libmysofa) makes the process freeze without errors or even text at all. Not sure if these issues are correlated but I'll probably have to report it, too. 🤔

[ABoredBunny]

@ThreeDeeJay I think what you are noticing is that the algorithm based ones focus more on volume(And parralex but that is likely already included in the multi field HRTF because it is measured). And also do very very close stuff stronger(like 10 cm) you can see it in the macrofx white paper they have a "Whisper zone". While the SCUT KEMAR HRTF only goes up to 20 cm. They might also exaggerate the effect.

If you are able to modify Sofa files you could probably replicate this(OpenAL soft just picks whatever field it is given, so you could use an unrealistic one). Another possible approach could be to see if a simulated HRTF provides better results than the measured one, simulating would let you simulate pretty much any distance including right up to the ear(Although you might need to clip the Responses that are inside the head model, and I don't know how OpenAL response to partial fields, in case it doesn't work you could replace the clipped points with unclipped points like 20 cm)

Mesh2HRTF as far as I know is "academicly" accurate and you can get a KEMAR 3D model like that. If you want to "waste" some compute time. They have different ways of setting up fields Including a blender plugin.

[kcat]

With regard to the ambi3rendering, I've noticed that using a smaller HRTF size seems to cause stronger bass with the near-field effect. The current data set I use is built with 128 sample points, and the bass boost is rather subtle. If I restrict it to 64, the bass boost is a bit more noticeable, and if I make it 32, it's even more noticeable. This is probably a result of shorter filter lengths not being able to filter lower frequencies properly.

But what @ABoredBunny says about the volume is true, too. The near-field effect is an effect distinct from distance attenuation. Those videos seems to be applying distance attenuation along with the near-field effect, and CMSS-3D MacroFX seems to apply it's own distance-related boost along with the near-field effect, whereas OpenAL Soft clamps distance attenuation like normal (according to the reference and max distance properties) and applies the near-field effect separate from distance attenuation.

[ThreeDeeJay]

I see. I guess that explains the fixed volume regardless of proximity within the circle here 🤔 And I've also been interested in synthesized HRTFs. That's why I've been trying to convert EAC and Mesh2HRTF SOFAs to MHR, hoping the flexibility of synthetic HRTFs like custom grid in EAC or merging multiple Mesh2HRTF radii SOFAs into a single MHR. Good tip on the KEMAR 3D model: I'll see if it can be used in Mesh2HRTF so perhaps we can have another "standard/default" HRTF we could modify and optimize as the tech improves. 👍