amazon-nova-samples icon indicating copy to clipboard operation
amazon-nova-samples copied to clipboard

firefox browser fix for sample rate incompatibility between usermedia and audiocontext

Open manikawnth opened this issue 8 months ago • 2 comments

… device and audiocontext

Issue #, if available: https://github.com/aws-samples/amazon-nova-samples/issues/78

Description of changes:

This fixes the firefox issue where the mismatched sample rates between the Audiocontext and input source throws and error and crashes the app

This fix allows firefox to use default system provided sample rate and programmatically down sample it to the model specific input config sample rate.

NOTE: This doesn't impact the existing behavior for Chrome browser as the sampling process is automatically done for Chromium based browsers as part of the AudioContext and hence the sampling ratio is alway 1 for such browsers.

Testing:

npm run build and num start the app, load the url in the firefox and test it.

Other remarks:

Firefox exhibits a behavior of feeding the audio from the speaker back to the microphone creating a self-talking loop. This is due to the fact that AudioPlayer.js has non-system-default output audio sampling rate. Ref: this.audioContext = new AudioContext({ "sampleRate": 16000 });

Hence it's recommended to perform the test using headphones only for Firefox. A separate PR will be raised for the output audio

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

manikawnth avatar Apr 10 '25 17:04 manikawnth

Firefox works fine, however, Chrome speech understanding is affected based on a quick functionality test. Behavior on Chrome shouldn't be changed.

prettyprettyprettygood avatar Apr 10 '25 18:04 prettyprettyprettygood

Hi @prettyprettyprettygood I completely isolated the chrome flow from the firefox. The flow from Chrome browser should be how it was originally in the sample. I've verified across all 3 major browsers and it's working so far.

Please review the changes and test again?

manikawnth avatar Apr 10 '25 21:04 manikawnth