net7mma_core
net7mma_core copied to clipboard
juliusfriedman
Providing multiple RTSP streams under single output from RtspServer
I've a bunch of input RTSP streams, and want to export them under single RTSP stream from RtspServer.
For example [rtsp://1, rtsp://2, rtsp://3] exported as rtsp://rtsp_server/uri.
Streams should be exported as a circular buffer with some delay between. Ex 1 stream exported for 30 seconds, then 2, etc.
- Is it possible with
RtspServer? Could you guide where to start? - Can I start multiple
RtspServerinstances to scale approach above? For example:
[rtsp://1, rtsp://2, rtsp://3] as circular buffer under rtsp://rtsp_server/uri1
[rtsp://4 rtsp://5, rtsp://6] as circular buffer under rtsp://rtsp_server/uri2
You want to do something like a camera tour where you tour between each stream?
It should be very easy to achieve provided the underlying streams are using the same codec, you can essentially just create a RtspStream and a timer and switch which underlying packets are going to the output RtspStream based on your desired logic, e.g. a counter to select from the proper underlying streams packets.
The logic would essentially on elapse of the timer attach to the events from the desired stream and enqueue them into the output stream for playback.
Give me a try and show me what you have tried and I can better assist you from there!
The code would be something like this:
someStream.RtpClient.RtpPacketReceieved += (s, p, tc) => outputStream.RtpClient.OnRtpPacketReceieved(p, tc);
Where someStream is the underlying stream you want to temporarily tour to and outputStream is the stream you have setup for playback.
Take a look @ https://github.com/juliusfriedman/net7mma_core/blob/master/UnitTests/Program.cs#L2125 (TestServer) for inspiration!
Well, may be it is dumb question.
I've tried to use TestServer method as a source.
int serverPort = 8000 + Media.Rtsp.RtspMessage.ReliableTransportDefaultPort;
System.Net.IPAddress serverIp =
Media.Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(
System.Net.Sockets.AddressFamily.InterNetwork);
Console.WriteLine("Server Starting on: {0}:{1}", serverIp, serverPort);
using (Media.Rtsp.RtspServer server = new Media.Rtsp.RtspServer(serverIp, serverPort)
{
Logger = new Media.Rtsp.Server.Loggers.RtspServerConsoleLogger(),
ClientSessionLogger = new Media.Rtsp.Server.Loggers.RtspServerConsoleLogger()
})
{
server.TryAddMedia(
new Media.Rtsp.Server.MediaTypes.RtspSource(
"R2_059",
"rtsp://8.15.251.101:1935/rtplive/R2_059"
)
);
server.Start();
// Wait for the server to start.
while (!server.IsRunning) System.Threading.Thread.Sleep(0);
Console.WriteLine("Listening on: " + server.LocalEndPoint);
while (true)
{
ConsoleKeyInfo keyInfo = Console.ReadKey(true);
if (keyInfo.Key == ConsoleKey.Q) break;
}
Console.WriteLine("Server Streamed : " + server.TotalStreamedBytes);
Console.WriteLine("Rtsp Sent : " + server.TotalRtspBytesSent);
Console.WriteLine("Rtsp Recieved : " + server.TotalRtspBytesRecieved);
Console.WriteLine("Stopping Server");
server.Stop();
Console.WriteLine("Server Stopped");
}
I can play original stream in VLC, but stream from server doesn't work in VLC.
When I try to connect to rtsp://192.168.0.85:8554/live/R2_059 via VLC I see this:
Received Invalid Message: * RTSP/0.0
Full log:
Server Starting on: 192.168.0.85:8554
Server Started @ 10/25/2023 3:08:31 AM
Listening on: 192.168.0.85:8554
Starting Stream: R2_059 Id=d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve: LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure.
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve - Begin
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve: LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure.
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve - Begin
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve: LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure.
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve - Begin
Adding Client: eb6753ba-bb47-4671-9ff0-6d7db93646e7
Accepted Client: eb6753ba-bb47-4671-9ff0-6d7db93646e7 @ 192.168.0.85:56194
10/25/2023 3:09:13 AM Added =True
Request=> OPTIONS rtsp://192.168.0.85:8554/live/R2_059 RTSP/1.0
CSeq: 2
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session=> eb6753ba-bb47-4671-9ff0-6d7db93646e7
RTSP/1.0 200
CSeq: 2
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Cache-Control: no-cache
Supported: play.basic,con.persistent,setup.playing
Server: ASTI Media Server RTSP\1.0
Request=> DESCRIBE rtsp://192.168.0.85:8554/live/R2_059 RTSP/1.0
CSeq: 3
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Accept: application/sdp
Session=> eb6753ba-bb47-4671-9ff0-6d7db93646e7
RTSP/1.0 200
CSeq: 3
Content-Type: application/sdp
Cache-Control: no-cache
Content-Base: rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/
Content-Length: 448
Content-Encoding: utf-8
Server: ASTI Media Server RTSP\1.0
v=0
o=ASTI-Media-Server 16781262518891554167 -1665481554806099096 IN IP4 192.168.0.85
s=ASTI-Streaming-Session R2_059
c=IN IP4 192.168.0.85
a=sdplang:en
a=range:npt=now-
a=control:*
t=0 0
m=video 0 RTP/AVP 97
a=rtpmap:97 H264/90000
a=fmtp:97 packetization-mode=1;profile-level-id=42C01E;sprop-parameter-sets=Z0LAHtoFB+wEQAAAAwBAAAAHo8WLqA==,aM48gA==
a=cliprect:0,0,240,320
a=framesize:97 320-240
a=framerate:15.0
a=control:trackID=1
Request=> SETUP rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/trackID=1 RTSP/1.0
CSeq: 4
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Transport: RTP/AVP;unicast;client_port=55634-55635
Session=> eb6753ba-bb47-4671-9ff0-6d7db93646e7
RTSP/1.0 200
CSeq: 4
Transport: RTP/AVP;source=192.168.0.85;unicast;client_port=55634-55635;server_port=30000-30001;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Session: 1634402757;timeout=30
Request=> PLAY rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 1634402757
Range: npt=0.000-
Session=> eb6753ba-bb47-4671-9ff0-6d7db93646e7
Media.Rtp.RtpClient-11202aa1-0e86-4303-9a04-47a977c10704@SendRecieve - Begin
Media.Rtp.RtpClient-11202aa1-0e86-4303-9a04-47a977c10704@ParseAndCompleteData - Remaining= 4
RTSP/1.0 200
Session: 1634402757;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.85/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/video;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Session:eb6753ba-bb47-4671-9ff0-6d7db93646e7 Attempting to complete previous mesage with buffer of 4 bytes.
Session:eb6753ba-bb47-4671-9ff0-6d7db93646e7 used 4 of buffer bytes
Request=> PLAY rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 1634402757
Range: npt=0.000-
Session=> eb6753ba-bb47-4671-9ff0-6d7db93646e7
RTSP/1.0 200
Session: 1634402757;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.85/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/video;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Received Invalid Message: * RTSP/0.0
For Session:eb6753ba-bb47-4671-9ff0-6d7db93646e7
Media.Rtp.RtpClient-11202aa1-0e86-4303-9a04-47a977c10704@ParseAndCompleteData - Remaining= 4
Session:eb6753ba-bb47-4671-9ff0-6d7db93646e7 Attempting to complete previous mesage with buffer of 4 bytes.
Session:eb6753ba-bb47-4671-9ff0-6d7db93646e7 used 4 of buffer bytes
Request=> PLAY rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 1634402757
Range: npt=0.000-
Session=> eb6753ba-bb47-4671-9ff0-6d7db93646e7
RTSP/1.0 200
Session: 1634402757;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.85/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/video;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Received Invalid Message: * RTSP/0.0
For Session:eb6753ba-bb47-4671-9ff0-6d7db93646e7
Media.Rtp.RtpClient-11202aa1-0e86-4303-9a04-47a977c10704@SendRecieve RtpSocket - SocketError = TimedOut lastOperation = 10/25/2023 3:09:13 AM taken = 00:00:00.0618179
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve: LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure.
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve - Begin
Media.Rtp.RtpClient-11202aa1-0e86-4303-9a04-47a977c10704@SendRecieve: LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure.
Media.Rtp.RtpClient-11202aa1-0e86-4303-9a04-47a977c10704@SendRecieve - Begin
Request=> TEARDOWN rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 6
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 1634402757
Session=> eb6753ba-bb47-4671-9ff0-6d7db93646e7
RTSP/1.0 200
Session: 1634402757;timeout=30
CSeq: 6
Server: ASTI Media Server RTSP\1.0
Session Inactive - 1634402757
Session Context Removed - 324543103@1634402757
Session Deactivated - 1634402757
Media.Rtp.RtpClient-11202aa1-0e86-4303-9a04-47a977c10704@SendRecieve - Exit
Adding Client: ce6e7802-b59d-4f0f-a506-668ea743d9ef
Request=> OPTIONS rtsp://192.168.0.85:8554/live/R2_059 RTSP/1.0
CSeq: 2
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session=> ce6e7802-b59d-4f0f-a506-668ea743d9ef
Accepted Client: ce6e7802-b59d-4f0f-a506-668ea743d9ef @ 192.168.0.85:56196
10/25/2023 3:09:24 AM Added =True
RTSP/1.0 200
CSeq: 2
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Cache-Control: no-cache
Supported: play.basic,con.persistent,setup.playing
Server: ASTI Media Server RTSP\1.0
Request=> DESCRIBE rtsp://192.168.0.85:8554/live/R2_059 RTSP/1.0
CSeq: 3
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Accept: application/sdp
Session=> ce6e7802-b59d-4f0f-a506-668ea743d9ef
RTSP/1.0 200
CSeq: 3
Content-Type: application/sdp
Cache-Control: no-cache
Content-Base: rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/
Content-Length: 448
Content-Encoding: utf-8
Server: ASTI Media Server RTSP\1.0
v=0
o=ASTI-Media-Server 16781262565443746052 -1665481508265769481 IN IP4 192.168.0.85
s=ASTI-Streaming-Session R2_059
c=IN IP4 192.168.0.85
a=sdplang:en
a=range:npt=now-
a=control:*
t=0 0
m=video 0 RTP/AVP 97
a=rtpmap:97 H264/90000
a=fmtp:97 packetization-mode=1;profile-level-id=42C01E;sprop-parameter-sets=Z0LAHtoFB+wEQAAAAwBAAAAHo8WLqA==,aM48gA==
a=cliprect:0,0,240,320
a=framesize:97 320-240
a=framerate:15.0
a=control:trackID=1
Request=> SETUP rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/trackID=1 RTSP/1.0
CSeq: 4
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Transport: RTP/AVP/TCP;unicast;interleaved=0-1
Session=> ce6e7802-b59d-4f0f-a506-668ea743d9ef
RTSP/1.0 200
CSeq: 4
Transport: RTP/AVP/TCP;source=192.168.0.85;interleaved=0-1;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Session: 151472816;timeout=30
Request=> PLAY rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 151472816
Range: npt=0.000-
Session=> ce6e7802-b59d-4f0f-a506-668ea743d9ef
Media.Rtp.RtpClient-042e3c94-3f8e-452a-9ab3-2e5f363fcf5a@SendRecieve - Begin
RTSP/1.0 200
Session: 151472816;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.85/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/video;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Media.Rtp.RtpClient-042e3c94-3f8e-452a-9ab3-2e5f363fcf5a@SendRecieve RtpSocket - SocketError = TimedOut lastOperation = 10/25/2023 3:09:24 AM taken = 00:00:00.0000014
042e3c94-3f8e-452a-9ab3-2e5f363fcf5aProcessFrameData - ParseAndHandleData
Media.Rtp.RtpClient-042e3c94-3f8e-452a-9ab3-2e5f363fcf5a@ProcessFrameData - raisedEvent for frameLength: 4 remainingInBuffer=0
Session:ce6e7802-b59d-4f0f-a506-668ea743d9ef Attempting to complete previous mesage with buffer of 4 bytes.
Session:ce6e7802-b59d-4f0f-a506-668ea743d9ef used 4 of buffer bytes
Request=> PLAY rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 151472816
Range: npt=0.000-
Session=> ce6e7802-b59d-4f0f-a506-668ea743d9ef
RTSP/1.0 200
Session: 151472816;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.85/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/video;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Received Invalid Message: * RTSP/0.0
For Session:ce6e7802-b59d-4f0f-a506-668ea743d9ef
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve: LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure.
Media.Rtp.RtpClient-8b095ebc-88d1-4d25-82cb-703ba2e41b02@SendRecieve - Begin
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions
042e3c94-3f8e-452a-9ab3-2e5f363fcf5aProcessFrameData - ParseAndHandleData
Media.Rtp.RtpClient-042e3c94-3f8e-452a-9ab3-2e5f363fcf5a@ProcessFrameData - raisedEvent for frameLength: 4 remainingInBuffer=0
Session:ce6e7802-b59d-4f0f-a506-668ea743d9ef Attempting to complete previous mesage with buffer of 4 bytes.
Session:ce6e7802-b59d-4f0f-a506-668ea743d9ef used 4 of buffer bytes
Request=> PLAY rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 151472816
Range: npt=0.000-
Session=> ce6e7802-b59d-4f0f-a506-668ea743d9ef
RTSP/1.0 200
Session: 151472816;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.85/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/video;ssrc=1358227F
Server: ASTI Media Server RTSP\1.0
Received Invalid Message: * RTSP/0.0
For Session:ce6e7802-b59d-4f0f-a506-668ea743d9ef
Media.Rtp.RtpClient-042e3c94-3f8e-452a-9ab3-2e5f363fcf5a@SendRecieve: LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure.
Media.Rtp.RtpClient-042e3c94-3f8e-452a-9ab3-2e5f363fcf5a@SendRecieve - Begin
Media.Rtp.RtpClient-042e3c94-3f8e-452a-9ab3-2e5f363fcf5a@ProcessFrameData - raisedEvent for frameLength: 183 remainingInBuffer=20
Request=> TEARDOWN rtsp://192.168.0.85:8554/live/d71eb5b9-09a9-48b8-b6a5-c2eda11e91f2/ RTSP/1.0
CSeq: 6
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: 151472816
Session=> ce6e7802-b59d-4f0f-a506-668ea743d9ef
042e3c94-3f8e-452a-9ab3-2e5f363fcf5aProcessFrameData - ParseAndHandleData
RTSP/1.0 200
Session: 151472816;timeout=30
CSeq: 6
Server: ASTI Media Server RTSP\1.0
https://github.com/juliusfriedman/net7mma_core/pull/24 doen't help, lines LengthInWordsMinusOne overflowed. Cannot store a number lower than 2 or higher than 0 in a 65535 structure. are gone, but log the same and no video playing in VLC.
Tried to rollback to commit without all my PVS-Studio commits (https://github.com/juliusfriedman/net7mma_core/commit/8ad1f3f9492b9caeb7b6633e026760d539f8103e), but problem still remains.
You only changed values for MP4 Boxes / Atoms, I think the values are actually in BigEndian, I have to double check the spec...
I am pretty sure there is something else wrong I will check the tests asap and let you know
I checked your code, everything seems okay but I am able to confirm over UDP there is either something wrong with your server or the parsing logic with the SDP, please do a wireshark capture of the RtspClientTest on your stream and submit it here so I can take a look when I have time.
I verified over TCP your stream works correctly with the RtspClient as well as the server through VLC.
Please take a look at why the issue occurs over UDP and let me know if you need my help, especially if you dont find anything on your side which causes the UDP session to fail.
It seems something broke in the RtspClient, its missing the Content of the Describe request sometimes...
If you add this logic (hack) it seems to work..
Describe:
response = SendRtspMessage(describe, out error, true, true, m_MaximumTransactionAttempts) ?? m_LastTransmitted;
if (!response.ContainsHeader(RtspHeaders.ContentLength)) goto Describe;
It's too late for me to check into this further but at least you know works over TCP.
Please try and review your changes in the RtspClient around SendDescribe to see why the RtspClient is failing.
I believe it might have something to do with the modified return but I cannot spot it easily.
Talk to you soon!
BTW, something is clearly happening with your server and UDP connections, it supplies and SDP the first time but then apparently stalls out and never finished the SDP for the subsequent describes...
In RtspClient @ StartPlaying you can also do something like this to verify it...
if (!string.IsNullOrEmpty(response.Body))
{
//Try to create a session description even if there was no contentType so long as one was not specified against sdp.
m_SessionDescription = new Sdp.SessionDescription(response.Body);
}
I can confirm our recent changes in RtspClient / RtpClient result in a StackOverflow exception which is exceedingly hard to track down without a stack trace. (It seems like Rtp over Udp is timing out and causing it somehow combined with MonitorProtocol)
I will continue to look into this when I have time but it probable will not be until the weekend.
Please let me know if you find it before me.
P.s. Tcp still works well (the problem only happens when using Udp)
You want to do something like a camera tour where you tour between each stream?
It should be very easy to achieve provided the underlying streams are using the same codec, you can essentially just create a RtspStream and a timer and switch which underlying packets are going to the output RtspStream based on your desired logic, e.g. a counter to select from the proper underlying streams packets.
The logic would essentially on elapse of the timer attach to the events from the desired stream and enqueue them into the output stream for playback.
Give me a try and show me what you have tried and I can better assist you from there!
The code would be something like this:
someStream.RtpClient.RtpPacketReceieved += (s, p, tc) => outputStream.RtpClient.OnRtpPacketReceieved(p, tc);Where someStream is the underlying stream you want to temporarily
tour toand outputStream is the stream you have setup for playback.Take a look @ https://github.com/juliusfriedman/net7mma_core/blob/master/UnitTests/Program.cs#L2125 (TestServer) for inspiration!
@juliusfriedman Could you please help me with this one more time?
I'm doing forwarding from input to output streams like this:
int serverPort = 8000 + Media.Rtsp.RtspMessage.ReliableTransportDefaultPort;
System.Net.IPAddress serverIp =
Media.Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(
System.Net.Sockets.AddressFamily.InterNetwork);
Console.WriteLine("Server Starting on: {0}:{1}", serverIp, serverPort);
using (Media.Rtsp.RtspServer server = new Media.Rtsp.RtspServer(serverIp, serverPort)
{
Logger = new Media.Rtsp.Server.Loggers.RtspServerConsoleLogger(),
ClientSessionLogger = new Media.Rtsp.Server.Loggers.RtspServerConsoleLogger()
})
{
server.Start();
while (!server.IsRunning) System.Threading.Thread.Sleep(0);
var input = new Media.Rtsp.Server.MediaTypes.RtspSource(
"R2_059",
"rtsp://8.15.251.101:1935/rtplive/R2_059",
Rtsp.RtspClient.ClientProtocolType.Tcp
);
// rtsp://8.15.251.101:1935/rtplive/R2_059 is H264 - MPEG-4 AVC (part 10) (h264) Video resolution: 320x240 Frame rate: 15
Media.Rtsp.Server.MediaTypes.RFC6184Media output = new Rtsp.Server.MediaTypes.RFC6184Media(320, 240, "OUT");
input.Start();
input.RtpClient.OutOfBandData += (s, d, o, l) =>
{
output.RtpClient.OnOutOfBandData(d, o, l);
};
input.RtpClient.RtpFrameChanged += (s, p, tc, f) =>
{
output.RtpClient.OnRtpFrameChanged(p, tc, f);
};
//inputStream1.RtpClient.RtpPacketSent += (s, p, tc) =>
//{
// outputStream.RtpClient.OnRtpPacketSent(p, tc);
//};
input.RtpClient.RtpPacketReceieved += (s, p, tc) =>
{
output.RtpClient.OnRtpPacketReceieved(p, tc);
};
input.RtpClient.RtcpPacketReceieved += (s, p, tc) =>
{
output.RtpClient.OnRtcpPacketReceieved(p, tc);
};
//inputStream1.RtpClient.RtcpPacketSent += (s, p, tc) =>
//{
// outputStream.RtpClient.OnRtcpPacketSent(p, tc);
//};
server.TryAddMedia(output);
Console.WriteLine("Listening on: " + server.LocalEndPoint);
while (true)
{
ConsoleKeyInfo keyInfo = Console.ReadKey(true);
if (keyInfo.Key == ConsoleKey.Q) break;
}
Console.WriteLine("Server Streamed : " + server.TotalStreamedBytes);
Console.WriteLine("Rtsp Sent : " + server.TotalRtspBytesSent);
Console.WriteLine("Rtsp Recieved : " + server.TotalRtspBytesRecieved);
Console.WriteLine("Stopping Server");
server.Stop();
Console.WriteLine("Server Stopped");
}
But looks like I do not understand your idea correctly and unable to play rtsp://192.168.0.77:8554/live/OUT in VLC.
What I'm doing wrong here?
It's just a bit more complex than it needs to be in order to work.
If you already have your output stream then its just a matter of calling packetize from the appropriate source..
You don't need the OutOfBandData and I personally would work with the RtspStream from the RtspServer rather than at the RtpClient level as it will decrease the complexity greatly.
All you need is an output stream:
Media.Rtsp.Server.MediaTypes.RtpAudioSink outputStream = new Rtsp.Server.MediaTypes.RtpAudioSink("Output", null, 0, 1, 8000);
Then from there you will attach to the output stream from the desired input stream like so:
someStream.RtpClient.RtpPacketReceieved += (s, p, tc) => outputStream.RtpClient.OnRtpPacketReceieved(p, tc);
This will cause the packets to begin to flow from someStream to outputStream.
Then upon a timer elapsing or otherwise you can easily stitch together different streams to create a tour provided your coded is the same for each stream (and the settings for the codec are relatively similar)
Remember to disconnect the events from the someStream when you are done with that particular instance so this way both streams are not flowing to the output at the same time.
If you need more help please let me know explicitly what.
Thank you again for your other contributions.
It's just a bit more complex than it needs to be in order to work.
If you already have your output stream then its just a matter of calling packetize from the appropriate source..
You don't need the OutOfBandData and I personally would work with the RtspStream from the RtspServer rather than at the RtpClient level as it will decrease the complexity greatly.
All you need is an output stream:
Media.Rtsp.Server.MediaTypes.RtpAudioSink outputStream = new Rtsp.Server.MediaTypes.RtpAudioSink("Output", null, 0, 1, 8000);Then from there you will attach to the output stream from the desired input stream like so:
someStream.RtpClient.RtpPacketReceieved += (s, p, tc) => outputStream.RtpClient.OnRtpPacketReceieved(p, tc);This will cause the packets to begin to flow from someStream to outputStream.
Then upon a timer elapsing or otherwise you can easily stitch together different streams to create a
tourprovided your coded is the same for each stream (and the settings for the codec are relatively similar)Remember to disconnect the events from the
someStreamwhen you are done with that particular instance so this way both streams are not flowing to the output at the same time.If you need more help please let me know explicitly what.
Thank you again for your other contributions.
Ok, I've done exactly as needed:
int serverPort = 8000 + Media.Rtsp.RtspMessage.ReliableTransportDefaultPort;
System.Net.IPAddress serverIp =
Media.Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(
System.Net.Sockets.AddressFamily.InterNetwork);
Console.WriteLine("Server Starting on: {0}:{1}", serverIp, serverPort);
using (Media.Rtsp.RtspServer server = new Media.Rtsp.RtspServer(serverIp, serverPort)
{
Logger = new Media.Rtsp.Server.Loggers.RtspServerConsoleLogger(),
ClientSessionLogger = new Media.Rtsp.Server.Loggers.RtspServerConsoleLogger()
})
{
server.Start();
while (!server.IsRunning) System.Threading.Thread.Sleep(0);
var output = new Rtsp.Server.MediaTypes.RtpAudioSink("Output", null, 0, 1, 8000);
// rtsp://8.15.251.101:1935/rtplive/R2_059 is H264 - MPEG-4 AVC (part 10) (h264) Video resolution: 320x240 Frame rate: 15
var input = new Media.Rtsp.Server.MediaTypes.RtspSource(
"R2_059",
"rtsp://8.15.251.101:1935/rtplive/R2_059",
Rtsp.RtspClient.ClientProtocolType.Tcp
);
input.Start();
input.RtpClient.RtpPacketReceieved += (s, p, tc) =>
{
output.RtpClient.OnRtpPacketReceieved(p, tc);
};
server.TryAddMedia(output);
Console.WriteLine("Listening on: " + server.LocalEndPoint);
while (true)
{
ConsoleKeyInfo keyInfo = Console.ReadKey(true);
if (keyInfo.Key == ConsoleKey.Q) break;
}
Console.WriteLine("Server Streamed : " + server.TotalStreamedBytes);
Console.WriteLine("Rtsp Sent : " + server.TotalRtspBytesSent);
Console.WriteLine("Rtsp Recieved : " + server.TotalRtspBytesRecieved);
Console.WriteLine("Stopping Server");
server.Stop();
Console.WriteLine("Server Stopped");
}
Query rtsp://192.168.0.77:8554/live/Output via VLC but still nothing.
Request=> PLAY rtsp://192.168.0.77:8554/live/59613cf9-49d5-4d0e-b2f9-51ee19e545ea/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: -1021074595
Range: npt=0.000-
Session=> 6a00caca-58d7-42f4-8d43-ad03d5670948
Media.Rtp.RtpClient-311663aa-2cc0-48b9-8347-a7776065eb3b@SendRecieve - Begin
RTSP/1.0 200
Session: -1021074595;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.77/live/59613cf9-49d5-4d0e-b2f9-51ee19e545ea/audio;ssrc=4601F9A6
Server: ASTI Media Server RTSP\1.0
Media.Rtp.RtpClient-311663aa-2cc0-48b9-8347-a7776065eb3b@SendRecieve RtpSocket - SocketError = TimedOut lastOperation = 10/25/2023 9:02:27 PM taken = 00:00:00.0000006
311663aa-2cc0-48b9-8347-a7776065eb3bProcessFrameData - ParseAndHandleData
Media.Rtp.RtpClient-311663aa-2cc0-48b9-8347-a7776065eb3b@ProcessFrameData - raisedEvent for frameLength: 4 remainingInBuffer=0
Session:6a00caca-58d7-42f4-8d43-ad03d5670948 Attempting to complete previous mesage with buffer of 4 bytes.
Session:6a00caca-58d7-42f4-8d43-ad03d5670948 used 4 of buffer bytes
Request=> PLAY rtsp://192.168.0.77:8554/live/59613cf9-49d5-4d0e-b2f9-51ee19e545ea/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.19 (LIVE555 Streaming Media v2016.11.28)
Session: -1021074595
Range: npt=0.000-
Session=> 6a00caca-58d7-42f4-8d43-ad03d5670948
RTSP/1.0 200
Session: -1021074595;timeout=30
CSeq: 5
Range: npt=now-
RTP-Info: url=rtsp://192.168.0.77/live/59613cf9-49d5-4d0e-b2f9-51ee19e545ea/audio;ssrc=4601F9A6
Server: ASTI Media Server RTSP\1.0
Received Invalid Message: * RTSP/0.0
For Session:6a00caca-58d7-42f4-8d43-ad03d5670948
Did you verify if you can play your source stream from the server by itself? If so please check out this code which should work with only minor changes:
//Make a new session description
Media.Sdp.SessionDescription compositeDescription = new Sdp.SessionDescription(0, "Bandit", "Composite");
//Add some required lines.
compositeDescription.Add(new Sdp.Lines.SessionConnectionLine()
{
ConnectionNetworkType = Sdp.Lines.SessionConnectionLine.InConnectionToken,
ConnectionAddressType = Sdp.SessionDescription.WildcardString,
ConnectionAddress = System.Net.IPAddress.Any.ToString()
});
//you need to access the session description from the media you wish to composite.
//Add the first
compositeDescription.Add(tcpStream.SessionDescription.MediaDescriptions.FirstOrDefault(), false);
//Add the second
compositeDescription.Add(sampleStream.SessionDescription.MediaDescriptions.FirstOrDefault(), false);
Media.Rtsp.Server.MediaTypes.RtpSource compositeSource = new Rtsp.Server.MediaTypes.RtpSource("Composite", compositeDescription);
var compositeContext = compositeSource.RtpClient.GetTransportContexts().First();
//Todo, Context.Synchronize or Context.Clone..
//Make the context a copy of the of the context to which it represents.
compositeContext.MinimumSequentialValidRtpPackets = 0;
compositeContext.AllowOutOfOrderPackets = true;
compositeContext.SynchronizationSourceIdentifier = tcpStream.RtpClient.GetTransportContexts().First().SynchronizationSourceIdentifier;
compositeContext.RemoteSynchronizationSourceIdentifier = tcpStream.RtpClient.GetTransportContexts().First().RemoteSynchronizationSourceIdentifier;
compositeContext.IsRtcpEnabled = false;
compositeContext.SendInterval = compositeContext.ReceiveInterval = Media.Common.Extensions.TimeSpan.TimeSpanExtensions.InfiniteTimeSpan;
compositeContext = compositeSource.RtpClient.GetTransportContexts().Last();
//
compositeContext.MinimumSequentialValidRtpPackets = 0;
compositeContext.AllowOutOfOrderPackets = true;
compositeContext.SynchronizationSourceIdentifier = sampleStream.RtpClient.GetTransportContexts().First().SynchronizationSourceIdentifier;
compositeContext.RemoteSynchronizationSourceIdentifier = sampleStream.RtpClient.GetTransportContexts().First().RemoteSynchronizationSourceIdentifier;
compositeContext.IsRtcpEnabled = false;
compositeContext.SendInterval = compositeContext.ReceiveInterval = Media.Common.Extensions.TimeSpan.TimeSpanExtensions.InfiniteTimeSpan;
int sharedClock = -1; ;
//Determine if packets or events will be handled
if (tcpStream.RtpClient.FrameChangedEventsEnabled)
{
compositeSource.RtpClient.FrameChangedEventsEnabled = true;
//Todo, Delcare a function so what comes in can be buffered if required.
tcpStream.RtpClient.RtpFrameChanged += (sender, frame, transportContext, final) =>
{
if (final)
{
if (sharedClock.Equals(-1))
{
sharedClock = frame.Timestamp;
}
else
{
sharedClock += frame.Timestamp - sharedClock;
}
compositeSource.RtpClient.OnRtpFrameChanged(frame, compositeSource.RtpClient.GetContextBySourceId(frame.SynchronizationSourceIdentifier), final);
}
};
}
else
{
compositeSource.RtpClient.FrameChangedEventsEnabled = false;
//Todo, Delcare a function so what comes in can be buffered if required.
tcpStream.RtpClient.RtpPacketReceieved += (sender, packet, transportContext) =>
{
if (packet.Marker)
{
if (sharedClock.Equals(-1))
{
sharedClock = packet.Timestamp;
}
else
{
sharedClock += packet.Timestamp - sharedClock;
}
}
else if (sharedClock.Equals(-1))
{
sharedClock = packet.Timestamp;
}
compositeSource.RtpClient.HandleIncomingRtpPacket(sender, packet, null);
};
}
//The same for the other stream
if (sampleStream.RtpClient.FrameChangedEventsEnabled)
{
//Todo, Delcare a function so what comes in can be buffered if required.
sampleStream.RtpClient.RtpFrameChanged += (sender, frame, transportContext, final) =>
{
if (final) compositeSource.RtpClient.OnRtpFrameChanged(frame, compositeSource.RtpClient.GetContextBySourceId(frame.SynchronizationSourceIdentifier), final);
};
}
else
{
//Todo, Delcare a function so what comes in can be buffered if required.
sampleStream.RtpClient.RtpPacketReceieved += (sender, packet, transportContext) =>
{
compositeSource.RtpClient.HandleIncomingRtpPacket(sender, packet, null);
};
}
//switch sources after x frames in events, retimestamp data or otherwise and send out.
//Finish the SDP
compositeDescription.Add(new Sdp.Lines.SessionConnectionLine()
{
ConnectionNetworkType = Sdp.Lines.SessionConnectionLine.InConnectionToken,
ConnectionAddressType = Sdp.SessionDescription.WildcardString,
ConnectionAddress = System.Net.IPAddress.Any.ToString()
});
//Indicate control to each media description contained can be attained from the main uri
//e.g. you can pause both streams at once.
compositeDescription.Add(new Sdp.SessionDescriptionLine("a=control:*"));
//Ensure the session members know they SHOULD only receive
compositeDescription.Add(new Sdp.SessionDescriptionLine("a=sendonly")); // this directive is for their `tools`
//that this a broadcast....
compositeDescription.Add(new Sdp.SessionDescriptionLine("a=type:broadcast"));
var md = compositeDescription.MediaDescriptions.First();
if (false.Equals(object.ReferenceEquals(md.ControlLine, null)))
{
md.Remove(md.ControlLine);
}
//How to control only this track, any valid grammar value should work but some libraries KISS/SUCK so...
md.Add(new Sdp.SessionDescriptionLine("a=control:trackID=1"));
//May already have a name line...
//md.Add(new Media.Sdp.Lines.SessionNameLine("Pics Stream"));
md = compositeDescription.MediaDescriptions.Last();
if (false.Equals(object.ReferenceEquals(md.ControlLine, null)))
{
md.Remove(md.ControlLine);
}
//How to control only that track
md.Add(new Sdp.SessionDescriptionLine("a=control:trackID=2"));
//May already have a name line.
//md.Add(new Media.Sdp.Lines.SessionNameLine("Bandit Stream"));
//Start the source now...
server.TryAddMedia(compositeSource);
System.Console.WriteLine("Started Composite");
//Could add third stream here...
break;
Please keep in mind in this example both streams flow to output at the same time. I didn't add logic here to stop the flow from either stream, just wanted to show how to create the composite stream.
P.s. This was already asked for and achieved before:
https://github.com/juliusfriedman/net7mma/blob/master/UnitTests/Program.cs#L2498
P.s. The above code will allow your users to select a stream through VLC or whatever player they are using.
If you don't need the stream to be selectable then you can skip having multiple tracks in the SDP and you can just use the code with a timer to switch out which source is attached to the compositeStream.
Let me know if you have any further questions!
ThreadExtensions.MinimumStackSize = 1 causes StackOverflow for me. Solved by setting one to 64 * 1024
My last attempt below. But it still causes artifacts in result video stream, hunging streaming, etc.
Result url is rtsp://<local ip address>:8554/live/carousel.
class TimedRingBuffer<T>
{
private readonly T[] buffer;
private readonly TimeSpan spanBetweenBufferIndexChange;
private readonly int timesToUseNoIndex;
private int currentIndex;
private DateTime? currentIndexStartDateTime;
public TimedRingBuffer(T[] buffer, TimeSpan spanBetweenBufferIndexChange)
{
if (buffer is null)
{
throw new ArgumentNullException(nameof(buffer));
}
if (buffer.Length == 0)
{
throw new ArgumentException("Sources should not be empty.", nameof(buffer));
}
if (timesToUseNoIndex < 0)
{
throw new ArgumentException($"Old index use time {timesToUseNoIndex} can't be < 0", nameof(timesToUseNoIndex));
}
this.buffer = buffer;
this.spanBetweenBufferIndexChange = spanBetweenBufferIndexChange;
this.currentIndex = 0;
}
public bool IsActiveNow(T item)
{
if (!currentIndexStartDateTime.HasValue)
{
this.currentIndexStartDateTime = DateTime.UtcNow;
}
if (currentIndexStartDateTime + spanBetweenBufferIndexChange <= DateTime.UtcNow)
{
this.currentIndex = (this.currentIndex + 1) % this.buffer.Length;
this.currentIndexStartDateTime = DateTime.UtcNow;
}
return Equals(item, buffer[currentIndex]);
}
}
static RtpSource CreateCompositeSource(RtspSource[] sources, string originator, string name, TimeSpan switchSourceAfterTimeSpan)
{
// Make a new session description
Sdp.SessionDescription compositeSession = new(0, originatorAndSession: originator, sessionName: name)
{
// Add some required lines.
new Sdp.Lines.SessionConnectionLine
{
ConnectionNetworkType = Sdp.Lines.SessionConnectionLine.InConnectionToken,
ConnectionAddressType = Sdp.SessionDescription.WildcardString,
ConnectionAddress = System.Net.IPAddress.Any.ToString()
}
};
foreach (var source in sources)
{
var mediaDescription =
source.SessionDescription.MediaDescriptions.FirstOrDefault()
?? throw new Exception($"Source '{source.Source}' missed media description.");
compositeSession.Add(mediaDescription, false);
}
var compositeSource = new RtpSource(name, compositeSession);
var compositeSourceTransportContexts =
compositeSource.RtpClient.GetTransportContexts().ToArray();
if (sources.Length != compositeSourceTransportContexts.Length)
{
throw new Exception(
$"Sources length ({sources.Length}) is not same as composite source contexts one ({compositeSourceTransportContexts.Length}).");
}
var timedRingBuffer = new TimedRingBuffer<RtspSource>(sources, switchSourceAfterTimeSpan);
for (int i = 0; i < sources.Length; i++)
{
var source = sources[i];
var compositeContext = compositeSourceTransportContexts[i];
// Make the context a copy of the of the context to which it represents.
SetupCompositeContextFromSource(source, compositeContext);
// Determine if packets or events will be handled.
ForwardCompositeEventsFromSource(source, compositeSource, timedRingBuffer.IsActiveNow);
}
// Switch sources after x frames in events, retimestamp data or otherwise and send out.
// Finish the SDP
compositeSession.Add(new Sdp.Lines.SessionConnectionLine
{
ConnectionNetworkType = Sdp.Lines.SessionConnectionLine.InConnectionToken,
ConnectionAddressType = Sdp.SessionDescription.WildcardString,
ConnectionAddress = System.Net.IPAddress.Any.ToString()
});
// Indicate control to each media description contained can be attained from the main uri
// e.g. you can pause both streams at once.
compositeSession.Add(new Sdp.SessionDescriptionLine("a=control:*"));
// Ensure the session members know they SHOULD only receive. This directive is for their `tools`
compositeSession.Add(new Sdp.SessionDescriptionLine("a=sendonly"));
// That this a broadcast....
compositeSession.Add(new Sdp.SessionDescriptionLine("a=type:broadcast"));
var compositeMediaDescriptions = compositeSession.MediaDescriptions.ToArray();
if (sources.Length != compositeMediaDescriptions.Length)
{
throw new Exception(
$"Sources length ({sources.Length}) is not same as composite media descriptions one ({compositeMediaDescriptions.Length}).");
}
for (int i = 0; i < compositeMediaDescriptions.Length; i++)
{
var compositeDescription = compositeMediaDescriptions[i];
var controlLine = compositeDescription.ControlLine;
if (controlLine is not null)
{
compositeDescription.Remove(controlLine);
}
// How to control only this track, any valid grammar value should work but some libraries KISS/SUCK so...
// compositeDescription.Add(new Sdp.SessionDescriptionLine($"a=control:trackID={i + 1}"));
// May already have a name line...
// compositeDescription.Add(new Sdp.Lines.SessionNameLine(sources[i].Name));
}
return compositeSource;
}
private static void ForwardCompositeEventsFromSource(RtspSource inputSource, RtpSource compositeSource, Func<RtspSource, bool> shouldForwardSource)
{
RtpClient sourceClient = inputSource.RtpClient, compositeClient = compositeSource.RtpClient;
bool isFrameChangedEventsEnabled = sourceClient.FrameChangedEventsEnabled;
compositeClient.FrameChangedEventsEnabled = isFrameChangedEventsEnabled;
if (isFrameChangedEventsEnabled)
{
sourceClient.RtpFrameChanged += (sender, frame, transportContext, final) =>
{
if (final && shouldForwardSource(inputSource))
{
var compositeTransportContext =
compositeClient.GetContextBySourceId(frame.SynchronizationSourceIdentifier);
compositeClient.OnRtpFrameChanged
(
frame,
compositeTransportContext,
final
);
}
};
}
else
{
sourceClient.RtpPacketReceieved += (sender, packet, transportContext) =>
{
if (shouldForwardSource(inputSource))
{
compositeClient.HandleIncomingRtpPacket(sender, packet, null);
}
};
}
sourceClient.RtcpPacketReceieved += (sender, packet, transportContext) =>
{
if (shouldForwardSource(inputSource))
{
compositeClient.HandleIncomingRtcpPacket(sender, packet, null);
}
};
}
private static void SetupCompositeContextFromSource(RtspSource inputSource, RtpClient.TransportContext compositeContext)
{
var transportContext = inputSource.RtpClient.GetTransportContexts().FirstOrDefault()
?? throw new Exception($"Source '{inputSource.Source}' has missed transport context.");
compositeContext.MinimumSequentialValidRtpPackets = 0;
compositeContext.AllowOutOfOrderPackets = true;
compositeContext.SynchronizationSourceIdentifier =
transportContext.SynchronizationSourceIdentifier;
compositeContext.RemoteSynchronizationSourceIdentifier =
transportContext.RemoteSynchronizationSourceIdentifier;
compositeContext.IsRtcpEnabled = false;
compositeContext.SendInterval =
compositeContext.ReceiveInterval =
Media.Common.Extensions.TimeSpan.TimeSpanExtensions.InfiniteTimeSpan;
}
static void TestServer()
{
System.Runtime.GCLatencyMode oldMode = System.Runtime.GCSettings.LatencyMode;
try
{
System.Runtime.GCSettings.LatencyMode = System.Runtime.GCLatencyMode.LowLatency;
int serverPort = 8000 + Rtsp.RtspMessage.ReliableTransportDefaultPort;
System.Net.IPAddress serverIp =
Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(
System.Net.Sockets.AddressFamily.InterNetwork);
Console.WriteLine("Server Starting on: {0}:{1}", serverIp, serverPort);
using (Rtsp.RtspServer server = new Rtsp.RtspServer(serverIp, serverPort)
{
Logger = new Rtsp.Server.Loggers.RtspServerConsoleLogger(),
ClientSessionLogger = new Rtsp.Server.Loggers.RtspServerConsoleLogger()
})
{
server.Start();
// Wait for the server to start.
while (!server.IsRunning) System.Threading.Thread.Sleep(0);
var inputSource1 = new RtspSource(
"R2_059",
"rtsp://8.15.251.101:1935/rtplive/R2_059",
Rtsp.RtspClient.ClientProtocolType.Tcp
);
inputSource1.Start();
var inputSource2 = new RtspSource(
"R2_052",
"rtsp://8.15.251.101:1935/rtplive/R2_052",
Rtsp.RtspClient.ClientProtocolType.Tcp
);
inputSource2.Start();
var compositeSource = CreateCompositeSource
(
new[] { inputSource1, inputSource2 },
originator: "nuSIM",
name: "carousel",
switchSourceAfterTimeSpan: TimeSpan.FromSeconds(5)
);
server.TryAddMedia(compositeSource);
inputSource1.RtpClient.ThreadEvents = false;
Console.WriteLine("Listening on: " + server.LocalEndPoint);
while (true)
{
ConsoleKeyInfo keyInfo = Console.ReadKey(true);
if (keyInfo.Key == ConsoleKey.Q) break;
}
Console.WriteLine("Server Streamed : " + server.TotalStreamedBytes);
Console.WriteLine("Rtsp Sent : " + server.TotalRtspBytesSent);
Console.WriteLine("Rtsp Recieved : " + server.TotalRtspBytesRecieved);
Console.WriteLine("Stopping Server");
server.Stop();
Console.WriteLine("Server Stopped");
}
}
finally
{
System.Runtime.GCSettings.LatencyMode = oldMode;
}
}
It seems like you had data flowing from both streams to the output source at the same time...
This is what caused the artifacts I assume, if you just leave one stream flowing through the compositeStream then you should not see any artifacts?
I think all that your missing is the removal of the events from the stream you don't want to display, It seems like why there are artifacts, because your sending data from both streams at the same time.
I dont' see how the TimedRingBuffer is useful yet, I don't think its needed because the Queue on each stream acts like a RingBuffer but I see your using it to switch the index of the stream so I will look into that as I don't have a lot of time right now, perhaps over the weekend.
I will check out your code but if you could provide me the link to 2 working streams it would be helpful for me to setup an example for you when I get time.
rtsp://8.15.251.101:1935/rtplive/R2_051 rtsp://8.15.251.101:1935/rtplive/R2_059
both are working.
Even if i do not do composite streaming at all, just
server.Start();
// Wait for the server to start.
while (!server.IsRunning) System.Threading.Thread.Sleep(0);
var inputSource1 = new RtspSource(
"R2_059",
"rtsp://8.15.251.101:1935/rtplive/R2_059",
Rtsp.RtspClient.ClientProtocolType.Tcp
);
inputSource1.Start();
server.TryAddMedia(inputSource1);
Wait a bit and you see artifacts in result stream. May be it is related to TCP proto, but VLC plays original stream without artifacts.
I doo see the artifacts, they seem to be reduced with larger buffer size...
server.TryAddMedia(new Media.Rtsp.Server.MediaTypes.RtspSource("R2_051", "rtsp://8.15.251.101:1935/rtplive/R2_051", Rtsp.RtspClient.ClientProtocolType.Tcp, 65540));
server.TryAddMedia(new Media.Rtsp.Server.MediaTypes.RtspSource("R2_059", "rtsp://8.15.251.101:1935/rtplive/R2_059", Rtsp.RtspClient.ClientProtocolType.Tcp, 65540));
I am seeing a lot of strange messages about buffer size exceeded, I will have to look into this to see exactly what is causing it.
I made a small push which now includes your streams, I will get to the composite stuff tonight or this weekend and if not sometimes next week.
I pushed up some code which swtiches between the 2 streams, you can setup the stream by pressing B after the server is running.
It seems to work okay for me, I need to look into why your cameras have so many artifacts in the stream, it seems like a bug on those cameras firmware or sometime of bandwidth exhaustion scenario...
I will let you know what I find ASAP.
I pushed up another small commit, it seems ThreadEvents can give your stream a little extra processing power and the artifacts seem to go away much quicker (but are still present during switching).
I will continue to look into this and let you know what I can find.
I found a small bug where packets were being missed when the buffer was empty, its a small change but seems to remove a lot of artifacts.
I will continue to look into why the remaining artifacts are present. (BTW if you use a really large buffer 10*1024) then it seems the artifacts clear up eventually.
I pushed up another small change but overall I think we will have better luck with the RtpVideoSink I will see if I can get an example working tomorrow, with that change we just have to add the frame and the Timestamp and Sequence numbers will be automatically calculated correctly without hacks...
I pushed up that change, it runs in VLC for hours at a time without any issues from what I can see, the artifacts come and go but I am sure we will track that down with time.
It should be trivial for your you to add streams to this example.
Please let me know if I can be of any other help!
BTW, also to reduce the artifacts when switching you can create a keyframe with sps and pps and send it right when the events are attached, you should get the sps and pps from the mediaDescription of the source stream, here is the example...
var frame = new RFC6184Media.RFC6184Frame(97);
var mediaDescription = currentStream.SessionDescription.MediaDescriptions.FirstOrDefault();
var spsPPs = mediaDescription.FmtpLine.ToString().Split(';').Last().Replace("sprop-parameter-sets=", string.Empty).Split(",");
frame.Packetize(Convert.FromBase64String(spsPPs[0]));
frame.Packetize(Convert.FromBase64String(spsPPs[1]));
frame.SynchronizationSourceIdentifier = compositeContext.SynchronizationSourceIdentifier;
compositeSource.Frames.Enqueue(frame);
If you do this then it helps to reduce the artifacts a bit right at the time of switching.
I pushed up some changes, it seems your camera has a lot of extra data on the TCP side, If you run the RtspClient test and attach to the Interleave events with the I key you will see there is always extra data.
I have verified this with Wireshark and although I am not sure where this is coming from yet or how VLC seems to deal with it so gracefully (without any debug messages in the log etc) it's something to be aware of for sure...
I will continue to look into this when I have time.
Still looking into this, it seems your camera is sending some other type of data (maybe sps and pps) which should be included in teh data stream.
I am not sure how live 555/vlc parses it so elegantly especially with all the strange malformed packets but its something I will continue to look into and see where I can get.
I will keep you updated.
(BTW Thanks for your continued work on the code)