amazon-chime-sdk-js
amazon-chime-sdk-js copied to clipboard
aws
Distorted audio with web audio
What happened and what did you expect to happen?
When the attendee opens another app, and browser goes to background the attendee audio getting distorted at the remote attendee side. Issue happens without noise suppression, But web audio flag is true.
Have you reviewed our existing documentation?
Reproduction steps
- Join meeting in chrome browser with web audio on.
- Open any other app and browser goes to background.
- Voice getting distorted.
Amazon Chime SDK for JavaScript version
3.1.0
What browsers are you seeing the problem on?
chrome
Browser version
101.0.4951.54
Meeting and Attendee ID Information.
No response
Browser console logs
none
@Yadukrish Just to clarify but when you say background do you mean in a mobile device or is this on desktop? Also you mention web audio, does that mean no audio issue if you turn off web audio?
The issue happens in desktop, when the user goes to any other applications while in the meeting. Issue started once we implemented noise suppression, haven't tested without web audio. Also the issue is there with noise suppression as well, at that time we are getting insufficient resources error message from observer.
Tested it again by disabling web audio issue is not there. So it seems like web audio issue.
Hi @Yadukrish,
Thanks for bringing the issue up. I can not reproduce the issue with our demo. Here are my testing steps:
- Joining from the Chrome (first attendee) and turn on the web audio and voice focus
- Join the same meeting(second attendee) with web audio/voice focus
- Mute the first attendee, and turn off the speaker on second attendee
- Move the meeting into background and open another application (Slack) on both Macs
- Second attendee keeps talking, the first attendee can hear clearly without voice distorting
- If I turn off the Web audio / switch first attendee and second attendee, the behavior is the same
Platform: First attendee: Chrome 101.0.4951.54, MacOS Monterey Second attendee: Chrome 101.0.4951.54, MacOS Big Sur
Could you try our demo to see if you can reproduce the same issue?
There is a known issue with macOS Safari: In macOS Safari, audio stops playing when I minimize or move the window to the background. Is this a known issue?
Just want to double check that this issue happens in Chrome? Could you provide your OS?
@Yadukrish Just to confirm but is the Chrome browser a receiver or a sender? In other words, is there any Safari client on the calls? We have only been able to repro the issue on MacOS Safari. Could you try our meeting demo to see if you have the same issue?
No there is no safari client, only chrome browser. Will try the meeting demo and test it.
Is there any progress or ideas what to check to sort this issue out? We are currently using your SDK in version 2.30.0 and we are reproducing the same issue on:
- Windows 11 devices with lates Chrome browser.
- Once observed also on Macbook Pro 2019 wiht some older MacOS (sorry don know detail)
Reproducing the issue: The issue most often appears (not always) when leaving browser tab with conference and working with some other application while conference is running (most often audio without video). After returning to the browser the issue is usually fixed. Unfortunatelly we are not able to reproduce it allways
Safari issue: There is that mentioned issue with inactive tab in Safari mentioned by @jing-chen1 however I consider this one maybe somehow related but not the same issue as we are discussing here. After leaving Safari tab there is compelte loss of the audio but after returning back it sometimes suffer with the similar audio distortion as discussed in this ticket
Distortion character: It sounds like distortion caused by audio buffer overflow/underflow where we are loosing some samples /frames of audio signal where result is discontinuous audio signal (hearable as countinuous popping and cracking in the audio)
Other observations: The issue was almost always reproduced with Voice Focuse (Noise supression) on. However today there was one testing session when issue appeared even after disabling of the Voice Focus, but it was still one session with no other audio manipulation (that's why we also want to check correct closing of Voice Focus processor and still can't rule Voice Focus out )
We have never managed to reproduce any of these issues with your demo application that we are running in the same version as the SDK version we use (2.30.0)
One more update from me. We have just managed to reproduce the bug on your demo application. As mentioned above the "webAudio" enabled is key. Steps we used to reproduce: 1)Run the demo app with "Web Audio" enabled. (We are also enabling simulcast and priority based downlink policy but I don't think these settings are relevant) 2)In mic settings enable Voice Focus feature 3)Leave the Chrome browser and go to Source Tre, Slack, IntelliJ Idea 4)Go back to the Chrome tab and then out of the Chrome a few times if needed
The distortion is hearable by other attendees or can be recorded (I haven tried to record with your demo app)
The upstreaming attendee with distortion had following setup: Chrome Browser 102.0.5005.63. Windows 11
I believe the Windows 11 is key because on this system I heard the issue very often (not always) and we managed to reproduce it. On Mac systems it was very rare. I have observed the issue on Mac rarely but never managed to reproduce it intentionally.
After some more digging I know the version of Mac where it was reproduced: macOS Catalina, version 10.15.7 MacBook Pro 13-inch, 2018) 2.3 GHz Quad-Core Intel Core i5 8 GB RAM
Chrome 101.0.4951.64
I haven't noticed anyone having this issue on MacOS Big Sur or Monterey yet
@LukasSmrcka @Yadukrish We believe this issue should be fixed with our 3.8.0 release, specifically the changes made in this PR: #2421 where we tuned a WebAudio parameter called latencyHint for Windows. Can you check and confirm if this resolves the issue for you?
@Yadukrish Can you confirm if Voice Focus was turned on when you last tested? Also, can you share your browser console logs for this repro?
@avinashmidathada voice focus aws chime issue.log Getting this error from sdk - noise suppression error - voiceFocusInsufficientResources - glitch in audio, noise suppression disabled temporarily.
We also have the same problem with a small part of our users. With noise cancellation disabled and the web audio flag set to true, we have problems reported by users related to audio: audio distortion/robotic, audio freezing after some time, audio too low...
In our case we have checked these meetings and all reports have in common that one of the users uses Windows and Chrome version 106.
this issue also seems to be related to robotic audio and Chrome version 106.
Amazon Chime SDK for JavaScript version 3.8.0
Meeting and Attendee ID Information
| meeting_id | participant_1_user_agent | participant_2_user_agent |
|---|---|---|
| afa93083-25cf-4b47-8527-b21281ca0706 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 |
| afa93083-25cf-4b47-8527-b21281ca0706 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 |
| 6c3f7a78-8983-43d4-ade1-a027a0930706 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 |
| 78fb8554-cf35-4e33-9717-7ae065cf0706 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 |
| 7ca409e9-5de6-4c14-bb71-7f4f6c8a0706 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.2 Safari/605.1.15 |
| 3e503948-2502-4f83-bd55-410ffec50706 | Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 | Mozilla/5.0 (Linux; Android 12; SAMSUNG SM-A715F) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/18.0 Chrome/99.0.4844.88 Mobile Safari/537.36 |
| 317134eb-80ec-4d0a-8158-8c1714cf0706 | Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 |
Also getting this error from observer when voice focus is turned on - "voiceFocusInsufficientResources" , then audio becomes choppy.
"Choppy" or "distorted" audio will happen in the case that Voice Focus is not able to keep up due to processing limitations of the device. If the observer is reporting that error, I would recommend disabling Voice Focus.
Please see:
- https://aws.github.io/amazon-chime-sdk-js/modules/amazonvoice_focus.html#observer-notifications
- https://aws.github.io/amazon-chime-sdk-js/modules/amazonvoice_focus.html#disabling-amazon-voice-focus-and-echo-reduction-and-switching-devices
In addition to ricchx@'s comment above, I'd like to clarify that there does seem to be an issue with Chromium's QoS state management on Windows 11. For some context, Win 11 has a performance optimization feature built in, wherein it takes away resources when an app is background-ed. For critical workloads such as media, there are provisions to maintain a high QoS for some threads so that there's no impact even if the app/browser is minimized.
We suspect that Chromium isn't managing this QoS state correctly and are planning to file a bug with them to further investigate and fix this issue. But, this might take quite a while as we'd need to depend on an external entity for the fix, so in the interim, the best solution is to not use voice focus or selectively disable it when the browser is backgrounded.
https://learn.microsoft.com/en-us/windows/application-management/enterprise-background-activity-controls windows allows to control resource allocation for backgrounded apps. If a user is always working in background while taking the call, that's something they could do by allowing browser to continue using resources even while backgrounded.
