On the streaming page, the camera device name is not shown at Firefox

[mustafaboleken]

On Firefox, navigator.mediaDevices.enumerateDevices() will return an empty label attribute value in the media device info if the respective permissions are not granted.

Environment

• Operating system and version: macOS 12.5.1
• Java version: 11
• Ant Media Server version: 2.4.3
• Browser name and version: Firefox 104.0.1

Steps to reproduce

1. Open streaming sample page.
2. Wait 3 seconds before allowing camera and microphone permission.
3. Check camera devices list.

Join community.antmedia.io and be a part of Ant Media Server Community.

[mustafaboleken]

Hi @rahul7827, you can track to issue from here.

[mekya]

Thank you. This issue is being resolved in this PR -> https://github.com/ant-media/StreamApp/pull/300

[rahul7827]

I think we can improve this solution further.

Issue is still there but a little different.

Since once we allow the camera and microphone's request of browser. It should display the device label in the next page reload. But its not happening.

Lets re-create at your end in incognito mode.

Step 1. Open amsserver.com/LiveApp/index.html in your browser. Step 2. Now check the device label (It will display Cam-01, Mic-01). This is fine because we have not granted the camera/mic request of browser. Step 3. Grant the camera/mic permission. Step 4. Reload the page.

Expected Behavior: We should have the device label at this time (But still Cam-01/Mic-01). Since we already granted this before page reload. Also you can observe after page reload your browser had not asked for camera/mic permission, which means permission are set and device label should display.

What i am assuming (I may be wrong) is.

Before completing the execution of navigator.mediaDevices.getUserMedia,function.

getDevices() function was executed. So we are getting empty device label.

Try this async/await in getDevices() function:

      `async getDevices() {
            await navigator.mediaDevices.getUserMedia({audio: true, video: true});
           await navigator.mediaDevices.enumerateDevices().then(devices => {  
  
     });`
	

This will resolve the issue permanently.

[mekya]

I've tested your solution and yes, you're right @rahul7827

[rahul7827]

Hi @mustafaboleken

I gone through this function initLocalStream() which is called when user is publisher.

 `initLocalStream() {
	this.checkWebRTCPermissions();

	// Get devices only in publish mode.
	this.getDevices(); // HERE I think we are calling this, before getting the browser permission. So it will always return blank label.
	this.trackDeviceChange();

	if (typeof this.mediaConstraints.video != "undefined" && this.mediaConstraints.video != false)
	{
		this.openStream(this.mediaConstraints, this.mode);	
	}
	else {
		// get only audio
		var media_audio_constraint = { audio: this.mediaConstraints.audio };
		this.navigatorUserMedia(media_audio_constraint , stream => {
			this.gotStream(stream);
		}, true)
	}
}`

I think these two function should be called right after getting the localStream, Instead of calling the getDevices() twice. It will also hit the callback Twice in this case.

Can we do this something like below

`initLocalStream() { this.checkWebRTCPermissions();

	// Get devices only in publish mode.

	if (typeof this.mediaConstraints.video != "undefined" && this.mediaConstraints.video != false)
	{
		this.openStream(this.mediaConstraints, this.mode);	
	}
	else {
		// get only audio
		var media_audio_constraint = { audio: this.mediaConstraints.audio };
		this.navigatorUserMedia(media_audio_constraint , stream => {
			this.gotStream(stream);
		}, true)
	}
          
         // Here we have to make sure the localStream is available.
        	this.getDevices();
	this.trackDeviceChange();
}`

OR

Other solution may be we can call these two functions here

gotStream(stream)
	{
		this.localStream = stream;
		if (this.localVideo) {
			this.localVideo.srcObject = stream;
		}
		
                this.getDevices();
		this.trackDeviceChange();
}

[mustafaboleken]

@rahul7827 thank you very much for your suggestions. I will optimize the process.

[mekya]

You're doing great @rahul7827 . Please let us know if we can help you for anything.

I think that @mustafaboleken will handle the rest.

[rahul7827]

Sure, Will let you know.

[mustafaboleken]

Hi @rahul7827 I try to optimize the process with respect to your comments but there are some problems. It took some time to respond because I try different ways to reduce complexity. Let me explain to you.

The first potential solution is adding the getDevices function at the end of the checkWebRTCPermissions function but this is not working because that openStream function and navigatorUserMedia functions are async, getDevices couldn't get device labels. Putting getDevices into these functions has no difference putting into gotStream, because they are generally called together.

The other potential solution is the removing getDevices function call inside initLocalStream to reduce calling twice but when we go that way, if user denied to camera&microphone access, list of video&audio sources are empty and I'm not sure it looks good. @mekya what is your comment about that? If user denied permission, we should shown empty list or labels like camera-0, camera-1 etc?

On the other hand, I think we need a callback when the user accepts the permissions, so we can call it once permissions are ready. As far as I know, browsers don't support such a solution right now. I found Permissions.query() method but unfortunately, it's not supported by most browsers and is labeled as experimental, so we shouldn't use it on production for now.

navigator.permissions.query({name: 'camera'}) .then((permission) => { console.log("camera state", permission.state); }).catch((error) => { console.log('Got error :', error); })

Thanks again for your time and effort :)

[rahul7827]

Hello @mustafaboleken

Thanks for the detailed workaround over this.

I think the second way would be appropriate

The other potential solution is the removing getDevices function call inside initLocalStream to reduce calling twice but when we go that way, if user denied to camera&microphone access, list of video&audio sources are empty and I'm not sure it looks good. @mekya what is your comment about that? If user denied permission, we should shown empty list or labels like camera-0, camera-1 etc?

As per my opinion (However you haven't asked to me, sorry for that 😄 ). There is no need to display audio/video labels if permissions are not granted.

Since without getting permission there is no use of the application for broadcast (Am i right ?).

So Instead of displaying blank labels we should give proper error like, "please allow camera/mic devices" along with a button which would retry to get the device permission and re-init the WebRTCAdaptor().

Apart from this we should also show an image how to unblock camera/mic permission if already blocked by user in their browser.

I will try to resolve this at my end hopefully weekend, if got success we can connect in meet call.

[mustafaboleken]

As per my opinion (However you haven't asked to me, sorry for that 😄 ). There is no need to display audio/video labels if permissions are not granted.

I forget to mention you, sorry for that. 😅 Actually, deciding between showing labels or empty lists is two different options and none of them are wrong I think. Because of that I wanted to get a third opinion. We can schedule a meet call, if you have things to show or discuss 😊

[mekya]

Hi Guys,

Done is better than perfect. Let's do the second solution and proceed. @mustafaboleken , are you available to make it or I can make it quickly because I'm just merging some PRs.

gotStream(stream)
	{
		this.localStream = stream;
		if (this.localVideo) {
			this.localVideo.srcObject = stream;
		}
		
                this.getDevices();
		this.trackDeviceChange();
}

[mustafaboleken]

Hi Guys,

Done is better than perfect. Let's do the second solution and proceed. @mustafaboleken , are you available to make it or I can make it quickly because I'm just merging some PRs.

gotStream(stream)
	{
		this.localStream = stream;
		if (this.localVideo) {
			this.localVideo.srcObject = stream;
		}
		
                this.getDevices();
		this.trackDeviceChange();
}

I will doing.

[mekya]

You're perfect @mustafaboleken

[rahul7827]

Done is better than perfect.

+1 👍

Because Perfect never gets done

[mekya]

Yesss, It's merged. Thank you guys for the collaboration @rahul7827 @mustafaboleken . You're great ;)