react-native-webrtc
react-native-webrtc copied to clipboard
react-native-webrtc
SessionDescription is NULL.
I'm currently trying to setup a RTCPeerConnection between a server and my react native client.
However when I set the remote description it fails with this error message:
{
name: 'SetRemoteDescriptionFailed',
message: 'SessionDescription is NULL.'
}
Code
const peer = new RTCPeerConnection({ iceServers: [...] })
const localOffer = await peer.createOffer();
await peer.setLocalDescription(localOffer);
// send local offer to server
// got sdp from server:
await peer.setRemoteDescription(new RTCSessionDescription({ sdp: sdp + "\n", type: "offer" }));
I have seen the duplicate issue #538 and tried adding \n at the end without success.
SDP
m=audio 50009 ICE/SDP
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
c=IN IP4 109.200.198.210
a=rtcp:50009
a=ice-ufrag:n+TW
a=ice-pwd:wzVglE/Kni4AIslkP7XUOE
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
a=candidate:1 1 UDP 4261412862 109.200.198.210 50009 typ host
I'm not sure if this is a bug or an issue on my side, but I'm glad for any help. Many thanks in advance and for taking the time to read this.
Platform information
- React Native version: 0.66.4
- Plugin version: 1.94.1
- OS: iOS 15.1.1
I've tried with and without the RTCSessionDescription for setLocalDescription and setRemoteDescription but for all cases it failed on setRemoteDescription.
Maybe the SDP is faulty? If yes, how do I check if the SDP is valid?
m=audio 50009 ICE/SDP a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87 c=IN IP4 109.200.198.210 a=rtcp:50009 a=ice-ufrag:n+TW a=ice-pwd:wzVglE/Kni4AIslkP7XUOE a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87 a=candidate:1 1 UDP 4261412862 109.200.198.210 50009 typ host
Ok, looks like the faulty SDP is the issue. After entering this in the browser console:
var peer = new RTCPeerConnection({ iceServers: [] });
peer.onsignalingstatechange = () => console.log("signal change", peer.signalingState);
peer.onicegatheringstatechange = () => console.log("ice gather");
peer.oniceconnectionstatechange = (e) => console.log("ice change", e);
peer.onicecandidateerror = (e) => console.log("ice error", e);
peer.onicecandidate = (e) => console.log("ice", e.candidate);
var { sdp, type } = await peer.createOffer();
await peer.setLocalDescription(new RTCSessionDescription({ sdp, type }));
var t = new RTCSessionDescription({type:"offer",sdp:`m=audio 50009 ICE/SDP
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
c=IN IP4 109.200.198.210
a=rtcp:50009
a=ice-ufrag:n+TW
a=ice-pwd:wzVglE/Kni4AIslkP7XUOE
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
a=candidate:1 1 UDP 4261412862 109.200.198.210 50009 typ host`})
await peer.setRemoteDescription(t)
I get this error message:
It would be nice if the library itself would show a better error message instead of SessionDescription is NULL, but thanks anyways!
PS: Do you know how I can solve it? 😄
Probably has no relevance but just wanted to share how I would solve it. I would correct the SDP as Discord (my testing webrtc server backend) does something weird with the SDP and strips important parts as noted in their webrtc article.
Ok, now I'm out of ideas. I've corrected the SDP to:
v=0
o=- 6054093392514871408 3 IN IP4 127.0.0.1
s=-
t=0 0
a=setup:passive
m=audio 50028 RTP/SAVPF 111
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
c=IN IP4 109.200.198.197
a=rtcp:50028
a=ice-ufrag:7iiG
a=ice-pwd:u3p03F0S5xP4fL4dVzlBjQ
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
a=candidate:1 1 UDP 4261412862 109.200.198.197 50028 typ host
a=rtcp-mux
a=mid:audio
b=AS:64
a=sendrecv
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10; useinbandfec=1
a=maxptime:60
and confirm that it works in the browser:
(in the browser I have to use RTCSessionDescription, but I removed it while using this react-native package)
However still get SessionDescription is NULL using this package without any other error indication.
Is there any way how I can enable debug mode to get more information about this error?
I've managed to get it working in the browser, however I'm still facing the same error code SessionDescription is NULL. with this package.
Notice: I've replaced all ip-addresses with
1.2.3.4for privacy reasons
react-native-webrtc local SDP:
v=0
o=- 5435327031003633143 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio
a=extmap-allow-mixed
a=msid-semantic: WMS A3AAB637-50CC-4FB2-90E9-712619ABF845
m=audio 47263 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126
c=IN IP4 1.2.3.4
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:3461351218 1 UDP 2122262783 1.2.3.4 61189 typ host generation 0 network-id 2 network-cost 10
a=candidate:2937932929 1 UDP 2122194687 1.2.3.4 52339 typ host generation 0 network-id 1 network-cost 10
a=candidate:2931479065 1 UDP 2122129151 1.2.3.4 59434 typ host generation 0 network-id 3 network-cost 10
a=candidate:2950599219 1 UDP 2122068735 1.2.3.4 54449 typ host generation 0 network-id 4 network-cost 50
a=candidate:3856669048 1 UDP 2122003199 1.2.3.4 50231 typ host generation 0 network-id 5 network-cost 50
a=candidate:2950599219 1 UDP 2121937663 1.2.3.4 62872 typ host generation 0 network-id 7 network-cost 50
a=candidate:2486661909 1 UDP 2121867007 1.2.3.4 60470 typ host generation 0 network-id 6 network-cost 50
a=candidate:1670411026 1 UDP 1685987071 1.2.3.4 52339 typ srflx raddr 1.2.3.4 rport 52339 generation 0 network-id 1 network-cost 10
a=candidate:2752489011 1 UDP 41820415 1.2.3.4 47263 typ relay raddr 1.2.3.4 rport 52339 generation 0 network-id 1 network-cost 10
a=candidate:3935066819 1 UDP 25042943 1.2.3.4 11585 typ relay raddr 1.2.3.4 rport 51349 generation 0 network-id 1 network-cost 10
a=candidate:3935066819 1 UDP 25042687 1.2.3.4 27001 typ relay raddr 1.2.3.4 rport 51352 generation 0 network-id 1 network-cost 10
a=ice-ufrag:3Lf9
a=ice-pwd:UFRzRaAREQ0XZJHrweozpdQg
a=ice-options:trickle renomination
a=fingerprint:sha-256 96:51:FF:A2:10:5F:3D:C2:6D:2B:2C:66:0D:1C:B2:48:82:BD:C0:7C:72:C8:4E:A9:B5:62:8F:2C:E3:C1:C2:13
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:102 ILBC/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3056027137 cname:0JvEzqml889UIqiz
a=ssrc:3056027137 msid:A3AAB637-50CC-4FB2-90E9-712619ABF845 6D530E5D-5353-4720-8F2B-89D6108710DF
a=ssrc:3056027137 mslabel:A3AAB637-50CC-4FB2-90E9-712619ABF845
a=ssrc:3056027137 label:6D530E5D-5353-4720-8F2B-89D6108710DF
", "protocol": "webrtc", "rtc_connection_id": "b0776bd8-7653-4c2f-8c7d-98c91482f906", "sdp": "v=0
o=- 5435327031003633143 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio
a=extmap-allow-mixed
a=msid-semantic: WMS A3AAB637-50CC-4FB2-90E9-712619ABF845
m=audio 47263 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126
c=IN IP4 1.2.3.4
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:3461351218 1 UDP 2122262783 1.2.3.4 61189 typ host generation 0 network-id 2 network-cost 10
a=candidate:2937932929 1 UDP 2122194687 1.2.3.4 52339 typ host generation 0 network-id 1 network-cost 10
a=candidate:2931479065 1 UDP 2122129151 1.2.3.4 59434 typ host generation 0 network-id 3 network-cost 10
a=candidate:2950599219 1 UDP 2122068735 1.2.3.4 54449 typ host generation 0 network-id 4 network-cost 50
a=candidate:3856669048 1 UDP 2122003199 1.2.3.4 50231 typ host generation 0 network-id 5 network-cost 50
a=candidate:2950599219 1 UDP 2121937663 1.2.3.4 62872 typ host generation 0 network-id 7 network-cost 50
a=candidate:2486661909 1 UDP 2121867007 1.2.3.4 60470 typ host generation 0 network-id 6 network-cost 50
a=candidate:1670411026 1 UDP 1685987071 1.2.3.4 52339 typ srflx raddr 1.2.3.4 rport 52339 generation 0 network-id 1 network-cost 10
a=candidate:2752489011 1 UDP 41820415 1.2.3.4 47263 typ relay raddr 1.2.3.4 rport 52339 generation 0 network-id 1 network-cost 10
a=candidate:3935066819 1 UDP 25042943 1.2.3.4 11585 typ relay raddr 1.2.3.4 rport 51349 generation 0 network-id 1 network-cost 10
a=candidate:3935066819 1 UDP 25042687 1.2.3.4 27001 typ relay raddr 1.2.3.4 rport 51352 generation 0 network-id 1 network-cost 10
a=ice-ufrag:3Lf9
a=ice-pwd:UFRzRaAREQ0XZJHrweozpdQg
a=ice-options:trickle renomination
a=fingerprint:sha-256 96:51:FF:A2:10:5F:3D:C2:6D:2B:2C:66:0D:1C:B2:48:82:BD:C0:7C:72:C8:4E:A9:B5:62:8F:2C:E3:C1:C2:13
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:102 ILBC/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3056027137 cname:0JvEzqml889UIqiz
a=ssrc:3056027137 msid:A3AAB637-50CC-4FB2-90E9-712619ABF845 6D530E5D-5353-4720-8F2B-89D6108710DF
a=ssrc:3056027137 mslabel:A3AAB637-50CC-4FB2-90E9-712619ABF845
a=ssrc:3056027137 label:6D530E5D-5353-4720-8F2B-89D6108710DF
remote SDP:
v=0
o=- 6054093392514871408 3 IN IP4 127.0.0.1
s=-
t=0 0
a=setup:passive
m=audio 50009 RTP/SAVPF 111
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
c=IN IP4 1.2.3.4
a=rtcp:50009
a=ice-ufrag:J11W
a=ice-pwd:bFlMqRh7KQmWVoLJbXgJQ6
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
a=candidate:1 1 UDP 4261412862 109.200.198.205 50009 typ host
a=rtcp-mux
a=mid:0
b=AS:64
a=sendrecv
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10; useinbandfec=1
a=maxptime:60
My log:
[WebRTC] negotiation needed [WebRTC] signal change have-local-offer [WebRTC] ice gather [WebRTC] ice candidates ... [WebRTC] got all candidates [WebRTC] get and send local sdp ... [WebRTC] got remote sdp [WebRTC] set remote sdp {"message": "SessionDescription is NULL.", "name": "SetRemoteDescriptionFailed"}
@Flam3rboy Hi! Having the same problem, have you solved it? Could you share with your solution?
Fixed in https://github.com/react-native-webrtc/react-native-webrtc/commit/576e8e877b96f401ca3a85e098fdeb767bb59359
I fixed my error by properly generating the SDP.
I used sdp-transform together with a custom lightweight transportation alternative to establish the call with much less overhead than the default 100kb SDP.
However, there were some issues with the sdp media tracks in my implementation that I was able to fix and resolve the issue.
I fixed my error by properly generating the SDP.
hi @SamuelScheit, can you share the solution for generating the sdp
@LightKnight3r It is probably not applicable for your project, because it is the way discord generates SDP but here you are:
import transform, { MediaDescription, SessionDescription } from "sdp-transform";
import { DiscordCall } from "./Call";
import { Participant } from "../Core/Call";
const codecs = [
{
name: "opus",
type: "audio",
priority: 1000,
payload_type: 111,
rtx_payload_type: null,
},
{
name: "H264",
type: "video",
priority: 1000,
payload_type: 102,
rtx_payload_type: 122,
},
{
name: "VP8",
type: "video",
priority: 2000,
payload_type: 96,
rtx_payload_type: 97,
},
{
name: "VP9",
type: "video",
priority: 3000,
payload_type: 98,
rtx_payload_type: 99,
},
];
export interface StreamInfo {
type: "video" | "audio";
ssrc: number;
rtx_ssrc?: number;
mid?: string;
user_id?: string;
}
export class WebRTC {
codecs = codecs;
pc!: RTCPeerConnection;
localSDP!: SessionDescription;
remoteMedia?: {
type: string;
port: number;
protocol: string;
payloads?: string | undefined;
} & MediaDescription;
audioSSRC = 0;
videoSSRC = 0;
rtxVideoSSRC = 0;
streams_inbound: StreamInfo[] = [];
negotiationPromise?: {
resolve: (value?: unknown) => void;
reject: (error?: Error) => void;
};
constructor(public call: DiscordCall) {}
truncateSDP = () => {
var o = new RegExp("^a=ice|a=extmap|a=fingerprint|opus|VP8|a=rtpmap:(96|97)", "i");
return {
sdp: this.pc
.localDescription!.sdp.split(/\r\n/)
.filter(function (e) {
return o.test(e);
})
.unique()
.join("\n"),
codecs: this.codecs,
};
};
init() {
this.destroy();
this.pc = new RTCPeerConnection({
bundlePolicy: "max-bundle",
// @ts-ignore
sdpSemantics: "unified-plan",
});
this.pc.addEventListener("negotiationneeded", async (e) => {
try {
await this.handleNegotiation();
} catch (error) {
console.error("[WebRTC] negotiationneeded error =>", error);
}
});
this.pc.addEventListener("icecandidateerror", (e) => {
this.call.setState("[WebRTC] Error: " + e);
});
this.pc.addEventListener("signalingstatechange", (e) => {
this.call.setState("[WebRTC] signalingState => " + this.pc?.signalingState);
});
this.pc.addEventListener("track", (e) => {
console.log("[WebRTC] track =>", e);
var stream = e.streams[0];
if (!stream) {
stream = new MediaStream();
stream.addTrack(e.track);
}
if (this.call.streams.some((x) => x.id === stream.id)) return;
this.call.streams.push(stream);
});
this.pc.addEventListener("connectionstatechange", (e) => {
this.call.setState("[WebRTC] connectionState => " + this.pc?.connectionState);
});
}
destroy() {
if (!this.pc) return;
try {
this.pc._unregisterEvents();
} catch (error) {}
this.pc.close();
this.pc = null as any;
}
setStream(stream: MediaStream) {
console.log("[WebRTC] setStream =>", stream);
var transceivers = this.pc.getTransceivers();
stream.getTracks().forEach((track) => {
var t = transceivers.shift();
if (t) t.direction = "sendrecv";
else t = this.pc.addTransceiver(track, { direction: "sendonly" });
t.sender.replaceTrack(track);
console.log("outgoing transceiver =>", t);
globalThis.t = t;
});
return new Promise((resolve, reject) => {
this.negotiationPromise = { resolve, reject };
});
}
setRemoteTruncatedSDP(sdp: string, video_codec: string, audio_codec: string) {
const { media } = transform.parse(sdp);
if (!media.length) return;
this.remoteMedia = media[0];
return this.handleNegotiation();
}
getRemoteSDP() {
if (!this.remoteMedia) throw new Error("Remote SDP not yet set");
var remoteSDP: SessionDescription = JSON.parse(JSON.stringify(this.localSDP));
// remote sdp is the simulated remote SDP that discord would send, if they followed the spec.
// We are constructing the SDP ourselves with the information we got over the websocket (ssrc, user id)
// We are using the local SDP as a template and replacing the information we got from discord
remoteSDP = {
...remoteSDP,
version: 0,
timing: {
start: 0,
stop: 0,
},
origin: {
address: "127.0.0.1",
ipVer: 4,
netType: "IN",
sessionId: "1420070400000",
sessionVersion: 0,
username: "-",
},
name: "-",
msidSemantic: {
semantic: "WMS",
token: "*",
},
};
remoteSDP.media.forEach((media) => {
// for each transceiver we generate an answer
// for all active transceivers, we return the required information (ssrc + active direction)
// set properties that are required for all transceivers
// candidates, connection, fingerprint, icepwd, iceufrag, port, protocol, rtcp, type (fmtp[],rtp[])
media.candidates = this.remoteMedia!.candidates;
media.connection = this.remoteMedia!.connection;
media.fingerprint = this.remoteMedia!.fingerprint;
media.icePwd = this.remoteMedia!.icePwd;
media.iceUfrag = this.remoteMedia!.iceUfrag;
media.port = this.remoteMedia!.port;
media.rtcp = this.remoteMedia!.rtcp;
media.setup = "passive";
media.protocol = "UDP/TLS/RTP/SAVPF";
media.rtcpMux = "rtcp-mux";
media.maxptime = 60;
if (media.type === "audio") {
// set audio specific properties
media.payloads = "111";
media.rtp = [
{
payload: 111,
codec: "opus",
rate: 48000,
encoding: 2,
},
];
media.fmtp = [
{
payload: 111,
config: "minptime=10;useinbandfec=1;usedtx=1",
},
];
media.rtcpFb = [
{
payload: 111,
type: "transport-cc",
},
];
media.ext = [
{
value: 1,
uri: "urn:ietf:params:rtp-hdrext:ssrc-audio-level",
},
{
value: 3,
uri: "http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01",
},
];
} else {
// set video specific properties
media.payloads = "102 122";
media.rtp = [
{
payload: 102,
codec: "H264",
rate: 90000,
},
{
payload: 122,
codec: "rtx",
rate: 90000,
},
];
media.bandwidth = [{ type: "AS", limit: 3000 }];
media.fmtp = [
{
payload: 102,
config: `level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f`,
},
{
payload: 122,
config: "apt=102",
},
];
media.rtcpFb = [
{
payload: 102,
type: "ccm",
subtype: "fir",
},
{
payload: 102,
type: "nack",
},
{
payload: 102,
type: "nack",
subtype: "pli",
},
{
payload: 102,
type: "goog-remb",
},
{
payload: 102,
type: "transport-cc",
},
];
}
// delete local only properties, that are not allowed/needed in the remote SDP
delete media.iceOptions;
delete media.ssrcGroups;
delete media.msid;
delete media.rtcpRsize;
const inbound = this.streams_inbound.find(
(x) => (x.mid == media.mid || x.mid == null) && x.type === media.type
);
if (media.ssrcs) {
// active transceivers that are being recorded and sent to discord
if (media.direction !== "sendrecv") media.direction = "recvonly";
delete media.ssrcs; // local ssrcs, not used for remote
} else if (inbound && media.direction !== "sendonly") {
// active transceivers that are rceived from discord
if (media.direction != "sendrecv") media.direction = "sendonly";
inbound.mid = media.mid;
media.ssrcs = [
{
id: inbound.ssrc,
value: inbound.user_id + "-" + inbound.ssrc,
attribute: "cname",
},
];
if (inbound.rtx_ssrc) {
// used for video
media.ssrcs.push({
id: inbound.rtx_ssrc!,
value: inbound.user_id + "-" + inbound.ssrc,
attribute: "cname",
});
media.ssrcGroups = [
{
semantics: "FID",
ssrcs: inbound.ssrc + " " + inbound.rtx_ssrc,
},
];
}
media.msid = `${inbound.user_id}-${inbound.ssrc} ${inbound.type === "audio" ? "a" : "v"}${
inbound.user_id
}-${inbound.ssrc}`;
console.log("attached inbound stream", inbound, media);
} else {
// inactive transceivers
media.direction = "inactive";
delete media.ssrcs;
}
});
return new RTCSessionDescription({ type: "answer", sdp: transform.write(remoteSDP) });
}
reverseDirection = (e: string) => {
switch (e) {
case "recvonly":
return "sendonly";
case "sendonly":
return "recvonly";
case "sendrecv":
return "sendrecv";
default:
return "inactive";
}
};
createUser = (userId: string, audioSSRC?: number, videoSSRC?: number, rtxSSRC?: number) => {
if (audioSSRC && !this.streams_inbound.find((x) => x.ssrc == audioSSRC)) {
console.log("createUser audio", userId, audioSSRC);
this.streams_inbound.push({
user_id: userId,
ssrc: audioSSRC,
type: "audio",
});
this.pc.addTransceiver("audio", { direction: "recvonly" });
}
if (videoSSRC && !this.streams_inbound.find((x) => x.ssrc == videoSSRC)) {
console.log("createUser video", userId, videoSSRC);
this.streams_inbound.push({
user_id: userId,
ssrc: videoSSRC,
rtx_ssrc: rtxSSRC,
type: "video",
});
this.pc.addTransceiver("video", { direction: "recvonly" });
}
if (!this.call.participants.has(userId)) {
this.call.participants.set(
userId,
new Participant(
{
user_id: userId,
muted: false,
ringing: false,
},
this.call
)
);
}
};
destroyUser = (userId: string, audioSSRC?: number, videoSSRC?: number) => {
this.call.participants.delete(userId);
const inbound = this.streams_inbound.find((x) => x.user_id == userId);
if (!inbound) return;
this.streams_inbound.remove(inbound);
this.pc
.getTransceivers()
.find((x) => x.mid === inbound.mid)
?.receiver.track.stop();
};
async handleNegotiation() {
console.log("[WebRTC] handleNegotiation");
const offer = await this.pc.createOffer({
iceRestart: false,
// offerToReceiveAudio: true,
});
this.localSDP = transform.parse(offer.sdp!);
console.log("[WebRTC] Local SDP", this.localSDP);
await this.pc.setLocalDescription(offer);
this.audioSSRC = this.localSDP.media.find(
(x) => x.type === "audio" && (x.direction === "sendonly" || x.direction === "sendrecv")
)?.ssrcs?.[0].id as number;
this.videoSSRC = this.localSDP.media.find(
(x) => x.type === "video" && (x.direction === "sendonly" || x.direction === "sendrecv")
)?.ssrcs?.[0].id as number;
this.rtxVideoSSRC = this.localSDP.media.find(
(x) => x.type === "video" && (x.direction === "sendonly" || x.direction === "sendrecv")
)?.ssrcs?.[2].id as number;
if (this.negotiationPromise) {
this.negotiationPromise.resolve();
this.negotiationPromise = undefined;
}
if (!this.remoteMedia) return;
const answer = this.getRemoteSDP();
console.log("[WebRTC] Remote SDP", transform.parse(answer.sdp!));
await this.pc.setRemoteDescription(answer);
}
}
@LightKnight3r It is probably not applicable for your project, because it is the way discord generates SDP but here you are:
import transform, { MediaDescription, SessionDescription } from "sdp-transform"; import { DiscordCall } from "./Call"; import { Participant } from "../Core/Call"; const codecs = [ { name: "opus", type: "audio", priority: 1000, payload_type: 111, rtx_payload_type: null, }, { name: "H264", type: "video", priority: 1000, payload_type: 102, rtx_payload_type: 122, }, { name: "VP8", type: "video", priority: 2000, payload_type: 96, rtx_payload_type: 97, }, { name: "VP9", type: "video", priority: 3000, payload_type: 98, rtx_payload_type: 99, }, ]; export interface StreamInfo { type: "video" | "audio"; ssrc: number; rtx_ssrc?: number; mid?: string; user_id?: string; } export class WebRTC { codecs = codecs; pc!: RTCPeerConnection; localSDP!: SessionDescription; remoteMedia?: { type: string; port: number; protocol: string; payloads?: string | undefined; } & MediaDescription; audioSSRC = 0; videoSSRC = 0; rtxVideoSSRC = 0; streams_inbound: StreamInfo[] = []; negotiationPromise?: { resolve: (value?: unknown) => void; reject: (error?: Error) => void; }; constructor(public call: DiscordCall) {} truncateSDP = () => { var o = new RegExp("^a=ice|a=extmap|a=fingerprint|opus|VP8|a=rtpmap:(96|97)", "i"); return { sdp: this.pc .localDescription!.sdp.split(/\r\n/) .filter(function (e) { return o.test(e); }) .unique() .join("\n"), codecs: this.codecs, }; }; init() { this.destroy(); this.pc = new RTCPeerConnection({ bundlePolicy: "max-bundle", // @ts-ignore sdpSemantics: "unified-plan", }); this.pc.addEventListener("negotiationneeded", async (e) => { try { await this.handleNegotiation(); } catch (error) { console.error("[WebRTC] negotiationneeded error =>", error); } }); this.pc.addEventListener("icecandidateerror", (e) => { this.call.setState("[WebRTC] Error: " + e); }); this.pc.addEventListener("signalingstatechange", (e) => { this.call.setState("[WebRTC] signalingState => " + this.pc?.signalingState); }); this.pc.addEventListener("track", (e) => { console.log("[WebRTC] track =>", e); var stream = e.streams[0]; if (!stream) { stream = new MediaStream(); stream.addTrack(e.track); } if (this.call.streams.some((x) => x.id === stream.id)) return; this.call.streams.push(stream); }); this.pc.addEventListener("connectionstatechange", (e) => { this.call.setState("[WebRTC] connectionState => " + this.pc?.connectionState); }); } destroy() { if (!this.pc) return; try { this.pc._unregisterEvents(); } catch (error) {} this.pc.close(); this.pc = null as any; } setStream(stream: MediaStream) { console.log("[WebRTC] setStream =>", stream); var transceivers = this.pc.getTransceivers(); stream.getTracks().forEach((track) => { var t = transceivers.shift(); if (t) t.direction = "sendrecv"; else t = this.pc.addTransceiver(track, { direction: "sendonly" }); t.sender.replaceTrack(track); console.log("outgoing transceiver =>", t); globalThis.t = t; }); return new Promise((resolve, reject) => { this.negotiationPromise = { resolve, reject }; }); } setRemoteTruncatedSDP(sdp: string, video_codec: string, audio_codec: string) { const { media } = transform.parse(sdp); if (!media.length) return; this.remoteMedia = media[0]; return this.handleNegotiation(); } getRemoteSDP() { if (!this.remoteMedia) throw new Error("Remote SDP not yet set"); var remoteSDP: SessionDescription = JSON.parse(JSON.stringify(this.localSDP)); // remote sdp is the simulated remote SDP that discord would send, if they followed the spec. // We are constructing the SDP ourselves with the information we got over the websocket (ssrc, user id) // We are using the local SDP as a template and replacing the information we got from discord remoteSDP = { ...remoteSDP, version: 0, timing: { start: 0, stop: 0, }, origin: { address: "127.0.0.1", ipVer: 4, netType: "IN", sessionId: "1420070400000", sessionVersion: 0, username: "-", }, name: "-", msidSemantic: { semantic: "WMS", token: "*", }, }; remoteSDP.media.forEach((media) => { // for each transceiver we generate an answer // for all active transceivers, we return the required information (ssrc + active direction) // set properties that are required for all transceivers // candidates, connection, fingerprint, icepwd, iceufrag, port, protocol, rtcp, type (fmtp[],rtp[]) media.candidates = this.remoteMedia!.candidates; media.connection = this.remoteMedia!.connection; media.fingerprint = this.remoteMedia!.fingerprint; media.icePwd = this.remoteMedia!.icePwd; media.iceUfrag = this.remoteMedia!.iceUfrag; media.port = this.remoteMedia!.port; media.rtcp = this.remoteMedia!.rtcp; media.setup = "passive"; media.protocol = "UDP/TLS/RTP/SAVPF"; media.rtcpMux = "rtcp-mux"; media.maxptime = 60; if (media.type === "audio") { // set audio specific properties media.payloads = "111"; media.rtp = [ { payload: 111, codec: "opus", rate: 48000, encoding: 2, }, ]; media.fmtp = [ { payload: 111, config: "minptime=10;useinbandfec=1;usedtx=1", }, ]; media.rtcpFb = [ { payload: 111, type: "transport-cc", }, ]; media.ext = [ { value: 1, uri: "urn:ietf:params:rtp-hdrext:ssrc-audio-level", }, { value: 3, uri: "http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01", }, ]; } else { // set video specific properties media.payloads = "102 122"; media.rtp = [ { payload: 102, codec: "H264", rate: 90000, }, { payload: 122, codec: "rtx", rate: 90000, }, ]; media.bandwidth = [{ type: "AS", limit: 3000 }]; media.fmtp = [ { payload: 102, config: `level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f`, }, { payload: 122, config: "apt=102", }, ]; media.rtcpFb = [ { payload: 102, type: "ccm", subtype: "fir", }, { payload: 102, type: "nack", }, { payload: 102, type: "nack", subtype: "pli", }, { payload: 102, type: "goog-remb", }, { payload: 102, type: "transport-cc", }, ]; } // delete local only properties, that are not allowed/needed in the remote SDP delete media.iceOptions; delete media.ssrcGroups; delete media.msid; delete media.rtcpRsize; const inbound = this.streams_inbound.find( (x) => (x.mid == media.mid || x.mid == null) && x.type === media.type ); if (media.ssrcs) { // active transceivers that are being recorded and sent to discord if (media.direction !== "sendrecv") media.direction = "recvonly"; delete media.ssrcs; // local ssrcs, not used for remote } else if (inbound && media.direction !== "sendonly") { // active transceivers that are rceived from discord if (media.direction != "sendrecv") media.direction = "sendonly"; inbound.mid = media.mid; media.ssrcs = [ { id: inbound.ssrc, value: inbound.user_id + "-" + inbound.ssrc, attribute: "cname", }, ]; if (inbound.rtx_ssrc) { // used for video media.ssrcs.push({ id: inbound.rtx_ssrc!, value: inbound.user_id + "-" + inbound.ssrc, attribute: "cname", }); media.ssrcGroups = [ { semantics: "FID", ssrcs: inbound.ssrc + " " + inbound.rtx_ssrc, }, ]; } media.msid = `${inbound.user_id}-${inbound.ssrc} ${inbound.type === "audio" ? "a" : "v"}${ inbound.user_id }-${inbound.ssrc}`; console.log("attached inbound stream", inbound, media); } else { // inactive transceivers media.direction = "inactive"; delete media.ssrcs; } }); return new RTCSessionDescription({ type: "answer", sdp: transform.write(remoteSDP) }); } reverseDirection = (e: string) => { switch (e) { case "recvonly": return "sendonly"; case "sendonly": return "recvonly"; case "sendrecv": return "sendrecv"; default: return "inactive"; } }; createUser = (userId: string, audioSSRC?: number, videoSSRC?: number, rtxSSRC?: number) => { if (audioSSRC && !this.streams_inbound.find((x) => x.ssrc == audioSSRC)) { console.log("createUser audio", userId, audioSSRC); this.streams_inbound.push({ user_id: userId, ssrc: audioSSRC, type: "audio", }); this.pc.addTransceiver("audio", { direction: "recvonly" }); } if (videoSSRC && !this.streams_inbound.find((x) => x.ssrc == videoSSRC)) { console.log("createUser video", userId, videoSSRC); this.streams_inbound.push({ user_id: userId, ssrc: videoSSRC, rtx_ssrc: rtxSSRC, type: "video", }); this.pc.addTransceiver("video", { direction: "recvonly" }); } if (!this.call.participants.has(userId)) { this.call.participants.set( userId, new Participant( { user_id: userId, muted: false, ringing: false, }, this.call ) ); } }; destroyUser = (userId: string, audioSSRC?: number, videoSSRC?: number) => { this.call.participants.delete(userId); const inbound = this.streams_inbound.find((x) => x.user_id == userId); if (!inbound) return; this.streams_inbound.remove(inbound); this.pc .getTransceivers() .find((x) => x.mid === inbound.mid) ?.receiver.track.stop(); }; async handleNegotiation() { console.log("[WebRTC] handleNegotiation"); const offer = await this.pc.createOffer({ iceRestart: false, // offerToReceiveAudio: true, }); this.localSDP = transform.parse(offer.sdp!); console.log("[WebRTC] Local SDP", this.localSDP); await this.pc.setLocalDescription(offer); this.audioSSRC = this.localSDP.media.find( (x) => x.type === "audio" && (x.direction === "sendonly" || x.direction === "sendrecv") )?.ssrcs?.[0].id as number; this.videoSSRC = this.localSDP.media.find( (x) => x.type === "video" && (x.direction === "sendonly" || x.direction === "sendrecv") )?.ssrcs?.[0].id as number; this.rtxVideoSSRC = this.localSDP.media.find( (x) => x.type === "video" && (x.direction === "sendonly" || x.direction === "sendrecv") )?.ssrcs?.[2].id as number; if (this.negotiationPromise) { this.negotiationPromise.resolve(); this.negotiationPromise = undefined; } if (!this.remoteMedia) return; const answer = this.getRemoteSDP(); console.log("[WebRTC] Remote SDP", transform.parse(answer.sdp!)); await this.pc.setRemoteDescription(answer); } }
Thanks for replying, i'll try this
in my case this can fix wrong sdp: https://stackoverflow.com/questions/66641204/sessiondescription-is-null-in-web-rtc-after-updating-chrome-to-latest-v-89-wor
extension RTCSessionDescriptionTransformable on RTCSessionDescription {
VCSessionDescription get toSessionDescription {
print("TYPE IN CONVERTER: ${type}");
return VCSessionDescription(
sdp: sdp?.replaceAll("a=extmap-allow-mixed\r\n", ""),
type: type == VCSessionDescriptionType.answer.name
? VCSessionDescriptionType.answer
: VCSessionDescriptionType.offer,
);
}
}
