overte icon indicating copy to clipboard operation
overte copied to clipboard

Some (stereo?) audio files don't spatialize

Open HifiExperiments opened this issue 1 year ago • 1 comments

if you are wearing headphones, sounds will spatialize, i.e. a sound to your right will sound louder in your right ear than your left. but this only works for some sounds.

this one spatializes: http://silverfish-freestuff.s3.amazonaws.com/TestStuff/SoundEntities/dtmf2.mp3

but these two don't: https://files.freemusicarchive.org/storage-freemusicarchive-org/music/no_curator/Scott_Holmes/Inspiring__Upbeat_Music/Scott_Holmes_-04-_Upbeat_Party.mp3 https://brainstormer.s3.us-west-2.amazonaws.com/Sounds/Balloon+Popping-SoundBible.com-1247261379.wav

this affects both sound entities and the old audio injectors:

{
  let sound = SoundCache.getSound(<sound URL>);
  let injector = Audio.playSound(sound, {
    position: MyAvatar.position,
    volume: 1.0,
    loop: true,
    pitch: 1.0
  });
}

this is probably related to this block of code in AudioClient.cpp:

                if (options.ambisonic) {
...
                    // spatialize into mixBuffer
                    injector->getLocalFOA().render(_localScratchBuffer, mixBuffer, HRTF_DATASET_INDEX,
                                                   qw, qx, qy, qz, gain, AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                } else if (options.stereo) {
...
                    // direct mix into mixBuffer
                    injector->getLocalHRTF().mixStereo(_localScratchBuffer, mixBuffer, gain,
                                                       AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                } else {  // injector is mono
...
                        // spatialize into mixBuffer
                        injector->getLocalHRTF().render(_localScratchBuffer, mixBuffer, HRTF_DATASET_INDEX,
                                                        azimuth, distance, gain, AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                    } else {
                        // direct mix into mixBuffer
                        injector->getLocalHRTF().mixMono(_localScratchBuffer, mixBuffer, gain,
                                                         AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                    }
                }

if the comments are to be believed, ambisonic and positional mono sounds spatialize, but stereo sounds only ever mix directly.

HifiExperiments avatar Apr 12 '24 04:04 HifiExperiments

I would agree. Stereo could be rendered as 2 Mono Channels so the azimuth, distance and gain would have to be computed for both channels. Right now Mono plays from the center of the entity. Stereo would be offset to the left and right +/- some value from the center. AudioHRTF.h and AudioHRTF.cpp would have to be updated so support stereo Rendering

1st-BrainStormer avatar Apr 12 '24 05:04 1st-BrainStormer