nextcloud
"checkPeerMedia" unmutes audio and video when a connection is established
When the MCU is used checkPeerMedia is used to unmute audio and video when some bytes were received according to the media track stats. However, it seems that in some cases this could lead to a peer being wrongly unmuted (in the receiving end; the original peer is still muted); for example, when joining a call in which a participant is muted.
Besides that I have also noticed that when a screen is shared the peer that receives the screen wrongly marks the video as available too; this is caused by the bytes received in the video track of the screen, which triggers an unmute event of video type and causes the normal video to be set as available.
@fancycode The screen sharing case can be fixed simply by not calling startCheckPeerMedia if the peer is a screen peer, because shared screens are not expected to be muted (in the sense of making the video not available, not audio muting), but I have no idea about why it happens for the microphone nor how it can be fixed in that case.
Note that the steps in scenario 1 sometimes work as expected, but it is caused by the audio off information sent when the connection is completed arriving after checkMediaPeer already unmuted the audio.
Steps to reproduce (Scenario 1)
- Setup Talk to use the MCU
- As user A, start a call
- As user A, disable microphone in the Talk UI
- As user B, join the call
Expected behaviour
User A is muted, and user B sees an indicator that user A is muted.
Actual behaviour
User A is muted, but user B does not see an indicator that user A is muted.
Steps to reproduce (Scenario 2)
- Setup Talk to use the MCU
- As user A, start a call with a camera available
- As user B, join the call
- As user A, disable camera in the Talk UI
- As user A, share local screen
Expected behaviour
User A video is disabled, and user B sees the avatar of user A.
Actual behaviour
User A video is disabled, but user B does not see the avatar of user A (a black video is shown instead) and user B sees an indicator that video of user A is enabled.
Besides what is described in the original report I have also found that the video is also unmuted by checkMediaPeer when a connection is established, exactly as it happens with the audio (that is, without needing to share the screen as described above). It is worth noting that this happens only with Chromium and recent Firefox versions; it does not happen with Firefox 52 for example, because in Firefox 52 it seems that no data is sent if the video is disabled.
Also note that this is not related to the SimpleWebRTC update; the same issue happened before it was merged. If checking this, however, note that checkMediaPeer does not work with latest Firefox and Chromium versions, as they deprecated the old callback based version of getStats (https://github.com/nextcloud/spreed/commit/31cc6de8e8a78440e585c5d4d3aed117a54ee110 and https://github.com/nextcloud/spreed/commit/a13c0a48159a3881c878401d18240992b0482415 would need to be applied on top of the parent commit of https://github.com/nextcloud/spreed/commit/cd90f8a43a33f0cf284d019c9379970c00f15065).
Like in the scenario 1 above, the steps in scenario 1-B sometimes work as expected, but it is caused by the video off information sent when the connection is completed arriving after checkMediaPeer already unmuted the video.
Steps to reproduce (Scenario 1-B)
- Setup Talk to use the MCU
- As user A, start a call
- As user A, disable camera in the Talk UI
- As user B, join the call
Expected behaviour
User A's video is disabled, and user B sees the avatar of user A (which indicates that user A's video is disabled).
Actual behaviour
User A's video is disabled, but user B sees a black video instead of the avatar of user A.
This is unrelated to Vue; it would be automatically once we stop using the data channels, but anyway I do not expect that to happen for 17 :-(
That is also what we see at the moment all the time, when people "send black video", right?
@danxuliu was https://github.com/nextcloud/spreed/pull/4686 your PR where we move away from data channels ?
This is still the case, right? Or is it better since simulcast and other things?