Is this a bug? Why are "one-way"-participants not receive the media stream after reloading the "two-way" participants?

[ghost]

Describe the bug

C-participant stops resive the media stream (video/audio) from B-participant when B-participant rejoin to the room. The same happens if B-participant entered the room later than the C-participant.

Scenario

A - initiator who opens the room (connection.open) B1, B2, ... - is "two-way" participants who communicate with the initiator and with each other in the room (connection.join) C1, C2, ... - observers is "one-way" participatnts who see and breathe all that is happening in the room (connection.join)

• A and B participant session is: {"audio": true, "video": true, "data": true}
• C-participant session is: {"audio": true, "video": true, "data": true, "oneway": true}

A-initiator opens the room. B-participants are connected to the room. A and B see and hear each other. The latter are connected to the room C - "one-way" participants. C-participants could only see and hear that happens in the room, but other participants (A and B) do not hear or see them.

Testing

But C and B participant get connection.peers from each other.

C-participant went to room the last and receive video/audio streem from B-participant. The parameters from the B-participant:

console.log (connection.peers['fs4zz2m7b4']);

PeerInitiator {extra: {…}, userid: "qwerty", streams: Array(1), channels: Array(1), connectionDescription: {…}, …}

If the B-participant reloads the page and rejoin (or enters the room after the C-participant). There is no media stream. Parameters from B-participant:

console.log (connection.peers['bvfj8f4cqen']);

PeerInitiator {extra: {...}, userid: "fs4zz2m7b4", streams: Array (0), channels: Array (1), connectionDescription: {...}, ...}

Turns out that, if B-participant rejoin then field streams: Array (0). Therefore, there is no media stream.

If C-participants reload the page, the Array (1) will contain data stream again

I note that the A-initiator has no such problems.

How to fix the problem?

What should be done so that after the reconnection of the B-participant (two-way) they are seen and heard by the C-participants ("one-way") automatically?

[ghost]

• RTCMultiConnection version: "3.5.0"
• RTCMultiConnection Socket.io Server version: "1.0.7"

[muaz-khan]

This feature isn't available in the latest rooms-implementation (since v3.4.7). I'm fixing demos step-by-step e.g. https://github.com/muaz-khan/RTCMultiConnection/commit/247a1b85072207da96910a02a15cc5b6ff4f8582 fixed password-protected-rooms demo.

Here is a relevant demo that will be fixed soon:

• https://rtcmulticonnection.herokuapp.com/demos/Multi-Broadcasters-and-Many-Viewers.html

You can temporarily use v3.4.6 though (which requires old signaling server a well):

• 724ead7cbb589a772b8bb41abd22acff3f700a5d/dist

[muaz-khan]

I'll fix following demo to work according to your needs (soon, in the next commit):

• https://rtcmulticonnection.herokuapp.com/demos/Multi-Broadcasters-and-Many-Viewers.html

[ghost]

Muaz, thank you!

Yes exactly. I use the Multi-Broadcasters and Many Viewers demo. I will wait for the correction of the new version - this will be correct.

Valeriy


пн, 22 окт. 2018 г. в 7:53, Muaz Khan <[email protected]>:

> This feature isn't available in the latest rooms-implementation (since
> v3.4.7). I'm fixing demos step-by-step e.g. 247a1b8
> <https://github.com/muaz-khan/RTCMultiConnection/commit/247a1b85072207da96910a02a15cc5b6ff4f8582>
> fixed password-protected-rooms demo.
>
> Here is a relevant demo *that will be fixed soon*:
>
>    -
>    https://rtcmulticonnection.herokuapp.com/demos/Multi-Broadcasters-and-Many-Viewers.html
>
> You can temporarily use v3.4.6 though (which requires old signaling server
> a well):
>
>    - 724ead7cbb589a772b8bb41abd22acff3f700a5d/dist
>    <https://github.com/muaz-khan/RTCMultiConnection/tree/724ead7cbb589a772b8bb41abd22acff3f700a5d/dist>
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/muaz-khan/RTCMultiConnection/issues/655#issuecomment-431739171>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AOzrF79HxKbvPNqSNoSq8t4ERZUaXIFyks5unU85gaJpZM4XynSh>
> .
>

[ghost]

Now I see that there have already been several updates. Maybe I missed something and some of them solve the problem?

While I noticed an interesting thing. If the B-participant reconnects the media source (webcam or microphone) using the following code:

function SwitchMedia(){
	connection.attachStreams.forEach(function(stream) { // disable local video track
		stream.getVideoTracks().forEach(function(track) {
			track.stop();
		});
	});
	
	connection.attachStreams.forEach(function(stream) { // remove video block with local stream
		stream.stop();
	});
	
	connection.removeStream({ // remove local video for all user
		video: true,
		audio: true
	});
	
	connection.addStream({
		audio: true,
		video: true
	});
}

In this case, the viewer - C-participant removes the old video stream and loads the new one. It is strange that it works. But, of course, not a solution to the problem as a whole.

[ghost]

This code that I put above does not always work correctly. Viever (C-participant ) at the command connection.removeStream does not always delete the stream. And the viewer shows the black window of the old stream and in a separate window shows the new stream. How can this be fixed?

[ghost]

Muaz, hello. The script has seriously changed. A camera switch appeared. Thank. But in this example, now there are no B-participants who can communicate with each other in the room.

It was with B-participants that was the problem. After the B-participant reconnect their medflows automatically appeared at the initiator. And vice versa - after the initiator restart - the B-participant and the C-participant automatically loaded the initiator’s media stream.

The problem was that C stops seeing B if B is reconnected. How to use your code to fix this problem? Maybe I do not see something. Tell me please.

[muaz-khan]

Functionality isn't changed. I merely removed the checkbox to enable same functionality through/from the broadcast button. Currently there are two kind of users:

1. Broadcasters who clicks Broadcast button to share their cams
2. Receivers who clicks to receive videos

There is no moderator currently. That's why I'm forcing autoCloseEntireSession=true. Because there is no moderator that's it doesn't matters if any of the broadcaster leaves.

Both receivers and broadcasters can use a looper function that keeps checking for new participants in that room and join them automatically. If you're a receiver then you will automatically receive new broadcaster's stream. If you're a broadcaster then you not only receive new broadcaster's stream but also share your own stream with him.

So please divide your target into following words:

1. There should be a super-admin or moderator (currently there is no such user)
2. There should be broadcasters (we've this system working)

3. There should be receivers (this system is also working)

Further steps:

1. If new broadcaster joins, all existing users must automatically receive video from him
2. If a broadcaster disconnects out of slow internet connection, he must reconnect to share stream again

etc.

[ghost]

Muaz, I looked carefully at your code and apologize for the last comment - in your example there are all three roles A, B and C.

I just had to guess that I needed to duplicate the tab in the browser. This was not clear, since the Join Broadcast button was previously used to turn on B-participants, not viewers. Maybe you should change the name of the buttons? )

[ghost]

In this example, the initiator of the room (and the viewers) see all the broadcasters. But broadcasters don't see each other.

Question to you, Muaz. How to make B-participants (broadcasters) communicate with each other, and not just with the initiator of the room?

[muaz-khan]

Yes. This is a bug. Because currently there is no room moderator and all users aren't connected to single room that's why we're not sharing participants with each other.

Solution is simple: make sure that all broadcasters connect to owner's room so that all broadcasters can see each other.

I'll fix that soon. It may require a little changes on signaling side, though.

[ghost]

Yes, broadcasters are connected to the room that the first broadcaster creates.

There is still a problem if several broadsters are switched on, and the viewer is connected last, several duplicate blocks appear on the screen in which the video is not loaded.

On the screenshot in the right window you can see black squares.

[ghost]

I tried to use complex mediaConstraints. I`m use a code that generates this structure:

	connection.mediaConstraints = {
		audio: {
			optional: [
				{sourceId: "qrIGlP/NItqEgHzuaHbGb1V2AYINU/4nHvwUIbcOH/0="},
				{googAutoGainControl: false}, 
				{googAutoGainControl2: false}, 
				{googEchoCancellation: false},
				{googEchoCancellation2: true},
				{googNoiseSuppression: false},
				{googNoiseSuppression2: true},
				{googHighpassFilter: true},
				{googTypingNoiseDetection: true},
				{googAudioMirroring: true},
				{googDisableLocalEcho: true}
			],
			mandatory: {
​​				deviceId: "qrIGlP/NItqEgHzuaHbGb1V2AYINU/4nHvwUIbcOH/0="
			​},
		},
		video: {
			optional: [
				{sourceId: "De1HS9gnLiTclio6ZoeUna82ttitPqu1KR7kpSLqdF4="},
				{facingMode: "user"} // 'user' or "application" for back camera
			], 
			mandatory: {
				deviceId: "De1HS9gnLiTclio6ZoeUna82ttitPqu1KR7kpSLqdF4=",
				maxHeight: 120,
​​				maxWidth: 160,
​​				minAspectRatio: 1.3333333333333333,
				minFrameRate: 5,
			}
		}
	};

For Firefox || Edge using these settings:

if (!!!navigator.webkitGetUserMedia) { 
	connection.mediaConstraints = {
		audio: {
​​			deviceId: "qrIGlP/NItqEgHzuaHbGb1V2AYINU/4nHvwUIbcOH/0="
		​},
		video: {
			deviceId: "De1HS9gnLiTclio6ZoeUna82ttitPqu1KR7kpSLqdF4=",
			maxHeight: 120,
​​			maxWidth: 160,
​​			minAspectRatio: 1.3333333333333333,
			minFrameRate: 5
		}
	}
}

navigator.mediaDevices.getUserMedia(connection.mediaConstraints ).then (function(stream) {
///..
}

The value of deviceId and sourceId is changed by a separate function. And the values of width, height, proportions and frequencies from a separate library. I give current values for an example.

This configuration works in Chrome!

Firefox switches cameras but does not apply settings. In the Edge everything is sad. When changing cameras local stream is not displayed, but the stream is sent to other broadcasters.

How to make the settings applied in all browsers?

[ghost]

Muaz, indeed, the broadcasts create different rooms (roomid = userid), because the connection.open is used. I do not fully understand how the initiator connects to the broadcasts. I thought that the participants are in different rooms are isolated from each other. That is what happens in the room does not see and does not hear the extraneous. Without you do not cope. (

[ghost]

I figured out a little why, when changing the webcam, the Height and Width parameters set in connection.mediaConstraints were not applied.

I looked in window.currentUserMediaRequest.streams. It turned out that when changing the camera, the parameters of the local stream do not change.

Earned after used:

connection.attachStreams.forEach(function(stream) { // delete old local video
	stream.getVideoTracks().forEach(function(track) {
		track.stop();
	});
	connection.onstreamended(stream);
});

which need to apply to:

navigator.mediaDevices.getUserMedia( selectConstraints() ).then (function(stream) { 
	document.getElementById(connection.userid).media.srcObject = stream;
	// ...
});

Normal work only in Chrome.

FireFox switches the camera but does not apply the settings. Edge switches the camera, sends the stream, but the local stream after the switch is not displayed.

[ghost]

Muaz, hi. I took your advice and tried to make it so that broadcasters could see each other. And after the reconnect, Broadcaster informed the Viewers that he was in the room and sent them a media stream.

1. In your code, I changed:

connection.socketCustomEvent = 'VideoService'+channelID;
connection.socketMessageEvent = channelID;

connection.session = {
	audio: true,
	video: true
};

2. I had to redo a little function of connecting participants. Participants again gather in one room:

//.............................................
//.................open and join room roles....
//.............................................
document.getElementById('open-or-join-room').focus();
document.getElementById('open-or-join-room').onclick = function() {
	connection.extra.broadcaster = true;
	selectConstraints();
	
	connection.checkPresence(roomid, function(isOwnerOnline) {
        if (isOwnerOnline == true) {
			connection.isInitiator = false;
			connection.join(roomid);
			afterConnectingSocket();
		} else {
			showRoomURL(roomid, channelID);
			connection.close();
			connection.closeSocket();
			connection.userid = roomid;
			connection.open(roomid);
			afterConnectingSocket();
			CheckParticipants();
		}
	});
	UIafterJoin();
}

//.............................................
//.................is autostart role...........
//.............................................
if ( params.autostart == 1) {// auto-join-room 
	connection.extra.broadcaster = true;
	startCheckingForOwnerPresence();
	afterConnectingSocket();
	UIafterJoin();	
}


//.............................................
//.................is View role................
//.............................................
if (params.roomid && roomid.length && isView) { // auto-join-room 
	document.getElementById('config').style.display = 'none';
	document.getElementById('chat-container').style.display ='none';
	
	connection.extra.broadcaster = false;
	connection.dontCaptureUserMedia = true;
	connection.session.oneway = true;
	
	startCheckingForOwnerPresence();
	afterConnectingSocket();
	UIafterJoin();
}

3. It was necessary to ensure that after join the Broadcaster sent the media stream to the viewers:

function afterConnectingSocket() {
	connection.socket.on(connection.socketCustomEvent, function(message) {
		console.log('custom message', message);
		
		if (message.giveAllParticipants && connection.isInitiator) { //is Initiator
			var participants = [];
			connection.getAllParticipants().forEach(function(pid) {
				participants.push({
					pid: pid,
					broadcaster: connection.peers[pid].extra.broadcaster === true
				});
			});
			connection.socket.emit(connection.socketCustomEvent, {
				participants: participants
			});
		}
		
		if (message.participants && !connection.isInitiator && connection.extra.broadcaster){ //is broadcasters connection with viewers
			message.participants.forEach(function(participant) {
				if (participant.pid === connection.userid) return;
				if (connection.extra.broadcaster && participant.broadcaster === true) return;
				
				connection.peers[participant.pid].addStream({//send stream to viewer
					audio: true,
					video: true
				});
			});
		}
	});
}

4. Automatic reconnect function for broadcasters and viewers:

var isOwnerPesenceCheckingOn = false;
var connectWithAllParticipantsOnce = false;

connection.onleave = function(event) {	
	var mediaElement = document.getElementById(event.userid);
	if (mediaElement && mediaElement.parentNode) {
		mediaElement.parentNode.removeChild(mediaElement);
		setTimeout(Resize,500,false); // Function for automatic recalculation of video block width
	}
	
	if (event.userid == connection.sessionid && !isOwnerPesenceCheckingOn ) {//connection to close for all participants if the Initiator left from the room
		
		console.info('Room is closed. Rechecking for room presence.');
		isOwnerPesenceCheckingOn = true;
		connectWithAllParticipantsOnce = true;
		connection.getAllParticipants().forEach(function(user) {
			connection.disconnectWith(user);
		});
		connection.close();
		connection.closeSocket();	
		connection.dontCaptureUserMedia = true; 
		startCheckingForOwnerPresence();
		afterConnectingSocket();
	}
};

function startCheckingForOwnerPresence() {
    connection.checkPresence(roomid, function(isOwnerOnline) {
        if (isOwnerOnline == true) {
            isOwnerPesenceCheckingOn = false;
			selectConstraints();
			connection.join(roomid, function(isRoomJoined, error) {
				if (isRoomJoined === false) {
					setTimeout(function() {
						startCheckingForOwnerPresence();
					}, 3000);
				}
			});
		
            return;
        }
        setTimeout(startCheckingForOwnerPresence, 2000); // recheck after 2 seconds
    });
}

If necessary, I can post the whole code. Do I need to do this here?

Participants connect after reconnect and give each other streams. Not all problems are solved yet.

If there are several viewers in the room, but after a couple of broadcasters come in, not all viewers load threads. Here are the error codes.

Viewer sdp-error DOMException: Failed to execute 'createAnswer' on 'RTCPeerConnection': PeerConnection cannot create an answer in a state other than have-remote-offer or have-local-pranswer. RTCMultiConnection.js:2940

setLocalDescription error DOMException: Failed to execute 'setLocalDescription' on 'RTCPeerConnection': Failed to set local answer sdp: Called in wrong state: kStable RTCMultiConnection.js:2936

broadcaster setRemoteDescription failed: DOMException: Failed to execute 'setRemoteDescription' on 'RTCPeerConnection': Failed to set remote answer sdp: Called in wrong state: kStable RTCMultiConnection.js:2775

While I understand what's the matter. I will be grateful for the help. Maybe you have a better - more stable solution?

[burak1489]

Did you find a solution for this problem?

[ghost]

No, I have not found a solution yet, and the problem remains.

[sayamk3004]

No, I have not found a solution yet, and the problem remains.

Did you find the solution yet?