In Safari and FF, no attendee shows up if joining a meeting with an empty audio device

[simmkyu]

Important: Please attach INFO-level logs with timestamps to the report from before the session started up until the point at which the unexpected behavior occured. For multi-participant calls, attach a log file for each participant if possible.

Meeting and attendee IDs: List the meeting and attendee IDs that the attached logs apply to. N/A

Timeline Outline the main actions the user performed and when: N/A

Describe the bug Describe the bug and the timestamp at which it happened relative to the timestamp in the log file.

Repro steps

1. In Safari or Firefox, assume you haven't allowed microphone in the permission dialog before. Also, you never called chooseAudioInputDevice with a non-null device.
2. Start a meeting.
3. An attendee will not show up.

Notes

• SDK creates an empty audio device using Web Audio API and joins the session in the following conditions.
  • chooseAudioInputDevice is never called before joining a meeting
  • chooseAudioInputDevice is called with null. Choosing null indicates no device.
• In Chrome, an attendee is present. Other attendees won't be able to hear you since you haven't selected any audio input device.
• In Firefox, an attendee is not present due to the no ice candidates were gathered error (Click here to see code).
• In Safari, an attendee is not present without any error.
• In the documentation and examples, we recommend choosing an audio input device before starting a meeting. If you choose an audio input device and allow microphone permission, this bug doesn't happen. An attendee will be present in all supported browsers.

Screenshots If applicable, add screenshots to help explain the problem.

Platform

• OS: macOS
• Browser: Safari and Firefox
• Browser Version: Safari 13.1 and Firefox 76
• Amazon Chime SDK GitHub Commit: 50b4ef8e73ad82e4735dab4ce6c9667f9c6a03b5

[johanderuijter]

Any updates on this? Our implementation allows users to join as an 'observer' (i.e. without audio/video inputs) but this bug currently means that doesn't work on firefox / safari?

[johanderuijter]

Some additional information in the case of safari, since we DO get an error when joining with en empty audio device:

[Info] 2020-07-03T13:20:05.944Z [INFO] ChimeMeetingLogs - serial group task AudioVideoStart/c9ab5dde-ed6d-4f2b-ab84-d9f2276f8641/a163acbc-b1d1-4b81-8d2f-ae7ed4af4e8c running subtask AudioVideoStart/c9ab5dde-ed6d-4f2b-ab84-d9f2276f8641/a163acbc-b1d1-4b81-8d2f-ae7ed4af4e8c/ReceiveAudioInputTask 
[Info] 2020-07-03T13:20:05.944Z [INFO] ChimeMeetingLogs - running task AudioVideoStart/c9ab5dde-ed6d-4f2b-ab84-d9f2276f8641/a163acbc-b1d1-4b81-8d2f-ae7ed4af4e8c/ReceiveAudioInputTask 
[Info] 2020-07-03T13:20:05.945Z [INFO] ChimeMeetingLogs - no audio device chosen, creating empty audio device 
[Info] 2020-07-03T13:20:05.945Z [INFO] ChimeMeetingLogs - requesting new audio device with constraint null 
[Log] Trace: 2020-07-03T13:20:05.951Z [ERROR] ChimeMeetingLogs - failed to get audio device for constraints null: The operation is not supported.
[Log] Trace: 2020-07-03T13:20:05.959Z [ERROR] ChimeMeetingLogs - unable to acquire audio device
[Warning] 2020-07-03T13:20:05.962Z [WARN] ChimeMeetingLogs - could not acquire audio input from current device 
[Warning] 2020-07-03T13:20:05.962Z [WARN] ChimeMeetingLogs - an audio input is not available 
[Info] 2020-07-03T13:20:05.963Z [INFO] ChimeMeetingLogs - AudioVideoStart/c9ab5dde-ed6d-4f2b-ab84-d9f2276f8641/a163acbc-b1d1-4b81-8d2f-ae7ed4af4e8c/ReceiveAudioInputTask took 19 ms 
[Info] 2020-07-03T13:20:05.963Z [INFO] ChimeMeetingLogs - serial group task AudioVideoStart/c9ab5dde-ed6d-4f2b-ab84-d9f2276f8641/a163acbc-b1d1-4b81-8d2f-ae7ed4af4e8c completed subtask AudioVideoStart/c9ab5dde-ed6d-4f2b-ab84-d9f2276f8641/a163acbc-b1d1-4b81-8d2f-ae7ed4af4e8c/ReceiveAudioInputTask 

A quick google seems to indicate that this error is thrown by safari when trying to create a buffer with an unsupported sample rate (https://webaudio.github.io/web-audio-api/#dom-baseaudiocontext-createbuffer)

Relate piece of code is i think this one: https://github.com/aws/amazon-chime-sdk-js/blob/c4f889e345af044283e230cf944a37e6f6964a78/src/devicecontroller/DefaultDeviceController.ts#L280-L285

[simmkyu]

Any updates on this? Our implementation allows users to join as an 'observer' (i.e. without audio/video inputs) but this bug currently means that doesn't work on firefox / safari?

No progress yet. SDK uses an audio input signal to notify the Chime server that the current attendee has joined a meeting. In case of no device selected, SDK joins a meeting with a fake audio input. Safari and FF are failing for different reasons.

As a workaround, could you join a meeting after muting an audio input?

// 1. Choose any audio input e.g. the first available device. Yo
const audioInputDeviceInfo = /* An array item from meetingSession.audioVideo.listAudioInputDevices */;
await meetingSession.audioVideo.chooseAudioInputDevice(audioInputDeviceInfo.deviceId);

// 2. Choose an audio output so this attendee, as an observer, can hear other attendees
const audioOutputDeviceInfo = /* An array item from meetingSession.audioVideo.listAudioOutputDevices */;
await meetingSession.audioVideo.chooseAudioOutputDevice(audioOutputDeviceInfo.deviceId);

// 3. Mute an audio input.
meetingSession.audioVideo.realtimeMuteLocalAudio();

// 4. Start a meeting. This attendee will be muted throughout the session.
meetingSession.audioVideo.start();

[simmkyu]

@johanderuijter

A quick google seems to indicate that this error is thrown by safari when trying to create a buffer with an unsupported sample rate (https://webaudio.github.io/web-audio-api/#dom-baseaudiocontext-createbuffer)

Thanks for the explanation. We pushed https://github.com/aws/amazon-chime-sdk-js/pull/555 to fix the unsupported sample rate issue.

When you choose the "null" audio device, SDK creates an AudioBuffer object with the AudioContext object's sample rate. In Safari, the AudioContext object is using the output device's sample rate. For example, SDK uses 16,000 Hz when Apple AirPods Pro is connected. In Safari, creating an AudioBuffer object with 16,000 Hz throws the The operation is not supported error.

Now, you should be able to join a meeting with the null audio device in Safari. However, you must allow the microphone permission before joining with null.

// 1. List audio input devices. The browser will show the permission dialog if you haven't allowed it before.
await meetingSession.audioVideo.listAudioInputDevices();

// 2. Choose the "null" audio input.
await meetingSession.audioVideo.chooseAudioInputDevice(null);

// 3. Choose an audio output so this attendee can hear other attendees.
const audioOutputDeviceInfo = /* An array item from meetingSession.audioVideo.listAudioOutputDevices */;
await meetingSession.audioVideo.chooseAudioOutputDevice(audioOutputDeviceInfo.deviceId);

// 4. Bind an audio element to hear other attendees
const audioElement = /* HTMLAudioElement object e.g. document.getElementById('audio-element-id') */;
meetingSession.audioVideo.bindAudioElement(audioElement);

// 5. Start a meeting. This attendee will be muted throughout the session.
meetingSession.audioVideo.start();

I'll keep this issue open because

• The problem persists in Firefox
• The user needs to allow the microphone before joining with null

[rubenfonseca]

Thank you for your hard work. However we are also very interested in using the Chime CDK with users that are just "observers", and right now the SDK is asking them to access their microphone which is extremely awkward. Would it ever be a way of allowing people to join a meeting only as "consumers"?

[xuesichao]

Hi everyone, although we have not fully resolved this issue yet, we have made some progress. Through our observations, we believe this issue is likely to be related to some restriction of browser.

As we mentioned before:

SDK uses an audio input signal to notify the Chime server that the current attendee has joined a meeting. In case of no device selected, SDK joins a meeting with a fake audio input.

In some cases, the browser seems to block the output of this fake audio input, causing the server to fail to receive the signal, which further prevents the user from joining the meeting.

Currently, there are two ways to fix it:

1. Before joining the meeting using the view-only mode, do NOT list or select any device. This will neither call the deviceLabelTrigger to trigger the device permission prompt nor cause the user to be unable to join the meeting.
2. Allow Autoplay for your Application website in Safari.
   • Open your application in the Safari app on your Mac.
   • Choose Safari > Settings for This Website. You can also Control-click in the Smart Search field, then choose Settings for This Website.
   • Hold the pointer to the right of Auto-Play, then click the pop-up menu and choose the option:
     • Allow All Auto-Play: Lets videos on this website play automatically.

We also added some new docs, check them below:

• How to implement a view-only mode in JS SDK
• How to fix the Safari Issue #474
• Join a meeting in view-only mode in React SDK

Let us know if this helps. We will keep working on this issue.

[kwhatmough]

Any update on the issue with Safari?

We are seeing issue https://github.com/aws/amazon-chime-sdk-component-library-react/issues/705 but it was closed as a duplicate of this issue which remains open. The workaround here does work but is less than ideal for our users. We would appreciate any update on this since the last status was from June 2021.

[etillson]

Also interested in whether the Safari issue has been resolved. Why would this prevent the user from appearing as an attendee in the roster? Is there another way to detect which users have joined/started the session?

[Ehbraheem]

I'm interested in the status of this issue as well.

[tkulis]

Hi there,

I noticed in my tests that in yours meeting demo app(tested on versions 3.6.0 & 3.10.0) (https://github.com/aws/amazon-chime-sdk-js/tree/main/demos/browser) this is not an issue. I am able to join in view-only mode flawless using iPhone 8 iOS 15.?

1. And I tried in my app , checking the meeting demo app logic=> didn't work
2. then following your suggestion/steps (https://aws.github.io/amazon-chime-sdk-js/modules/apioverview.html#implement-a-view-onlyobserverspectator-experience)=> didn't work,
3. and at the end installing amazon-chime-sdk-component-library-react and following steps there (https://aws.github.io/amazon-chime-sdk-component-library-react/?path=/story/quick-starts--page) => didn't work

Do you do something special in this meeting demo app what I cannot find 😀?

[xuesichao]

@tkulis , I don't think we have any. Did you make sure your browser device permission status was reset before each test?