AlexxIT
Issues with simple (?) setup for WebRTC stream with backchannel
Hi @AlexxIT - I am sure I am doing something wrong here but what should be a simple setup has got me bewildered and I wonder if you can help.
Hardware: an RTSP camera which produces H264 video, AAC audio, PCMU backchannel (let's say rtsp://1.2.3.4/stream). I need to have only a single connection to the camera's RTSP endpoint.
What I want from go2rtc:
- A WebRTC stream which copies video & backchannel, and transcodes audio to Opus.
If I do something like:
streams:
camera:rtsp://1.2.3.4/stream
Then WebRTC video and backchannel works, but audio doesn't - fair enough since there's no transcoding to Opus. So now if I try:
streams:
camera:ffmpeg:rtsp://1.2.3.4/stream#video=copy#audio=opus#backchannel=1
Then WebRTC video and audio work, but backchannel doesn't work (WebRTC track is absent). So now if I try:
streams:
camera:
- ffmpeg:rtsp://1.2.3.4/stream#video=copy#audio=opus
- rtsp://1.2.3.4/stream
Then now everything "works" (WebRTC video & audio & backchannel) but I end up with two RTSP connections to the camera.
Also tried this, but this also gives WebRTC video & audio, but no working backchannel:
streams:
base_stream:rtsp://1.2.3.4/stream
camera:
- ffmpeg:rtsp://1.2.3.4/stream#video=copy#audio=opus
- streams:base_stream
So the question is: how do I get a WebRTC stream with video copied, backchannel copied and audio transcoded to Opus (i.e. so it's WebRTC compatible) without go2rtc making two RTSP connections to the camera?
Thank you so much!
-Adrian
@adriancable
streams:
my_camera:
- rtsp://127.0.0.1:1234/stream
- ffmpeg:my_camera#audio=opus
@seydx - thanks, that is better. But this now surfaces another issue. In addition to the WebRTC stream (which now works), I also want another consumer to connect to the RTSP stream (like an NVR). However, if the WebRTC stream is live and the other consumer connects (to e.g. rtsp://localhost:8554/my_camera), it looks like the streams aren't being multiplexed on the go2rtc side correctly, because I get 'bad cseq' errors and the stream blows up:
09:42:13.756 DBG [exec] [rtsp @ 0xac7060000] RTP: PT=60: bad cseq 44d5 expected=e5e5 09:42:13.768 DBG [exec] [rtsp @ 0xac7060000] RTP: PT=61: bad cseq 44ca expected=c611
I have also tried something like:
streams:
my_camera:
- rtsp://1.2.3.4/stream
- ffmpeg:my_camera#audio=opus
recording_stream:
- ffmpeg:my_camera#audio=opus
with the WebRTC stream connecting to my_camera and the 'NVR' connecting to rtsp://localhost:8554/recording_stream. But this also blows up with the same 'bad cseq' errors once the NVR connects. (If the NVR connects first, that's OK, but then when the WebRTC stream connects it then blows up with the 'bad cseq' errors.)
Note I don't get the 'bad cseq' errors with simultaneous WebRTC and NVR connections if the config is just
streams:
my_camera:ffmpeg:rtsp://1.2.3.4/stream#audio=opus
But then we're back to the WebRTC backchannel not working.
Any ideas?
This happens because go2rtc triggers a reconnect. When the second consumer wants different tracks than already established ones, go2rtc reconnects and this usually causes FFmpeg to throw errors like bad cseq etc. (because cseq is no longer sequential and was reset by the reconnect).
To verify this, you can try the following:
- Connect to the stream via WebRTC (with backchannel)
- Then connect to the RTSP stream with your NVR or something similar using:
rtsp://localhost:8554/my_camera
The second consumer, in this case the NVR, should be able to display the video without any issues...
You can handle audio tracks via query parameters, for example:
rtsp://localhost:8554/my_camera?video&audio=opus&backchannel=1
Here the same tracks are requested that were already created by the WebRTC connection, so go2rtc doesn't need to reconnect to renegotiate the tracks.
Oh, one more thing:
if the second consumer also wants the backchannel, this will trigger a reconnect. go2rtc currently offers NO way for two or more consumers to share the backchannel...
There's a PR from me for this that you can use: https://github.com/AlexxIT/go2rtc/pull/1953
This bypasses the reconnect caused by the second consumer also wanting the backchannel, because we create an audio mixer... This means an unlimited number of consumers can share the same backchannel and multiple consumers can speak into the backchannel simultaneously.
Hi @seydx - I did see your PR when digging into my issue earlier. It looks great but at this time I don't need multiple consumers for the backchannel. Unfortunately, I still have the same issue (or maybe a worse issue).
After the WebRTC stream is established and working, when the 'NVR' (which does not use the backchannel) connects to rtsp://localhost:8554/my_camera?video&audio=opus&backchannel=1 I now get:
10:15:18.726 DBG [exec] [rtsp @ 0x7fcc58000] RTP: PT=61: bad cseq 5947 expected=363e
10:15:18.824 DBG [exec] [vf#0:0 @ 0x7fccb0540]
10:15:18.824 DBG [exec] 4497611 frame duplication too large, skipping
10:15:18.846 DBG [exec] [vf#0:0 @ 0x7fccb0540] 4497613 frame duplication too large, skipping
10:15:18.851 DBG [exec] [vf#0:0 @ 0x7fccb0540] 4497617 frame duplication too large, skipping
10:15:18.888 DBG [exec] [vf#0:0 @ 0x7fccb0540] 4497621 frame duplication too large, skipping
I'm super appreciative of your help. Any other ideas?
hm it seems you have a video filter in your ffmpeg config? can you show me your config.yaml
@seydx - sorry for causing you confusion. The vf errors were a red herring. This is my config, verbatim (JSON format):
{
"log": {
"format": "text",
"level": "debug"
},
"api": {
"listen": "127.0.0.1:1984",
"origin": "*"
},
"rtsp": {
"listen": "127.0.0.1:8554"
},
"streams": {
"camera": [
"rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif",
"ffmpeg:camera#audio=opus"
]
}
}
This is the output produced from go2rtc. In this minimal example, I actually have only one consumer connecting but the 'bad cseq' errors happen immediately after that.
10:38:13.796 INF go2rtc platform=darwin/arm64 revision=mod.aa0ece2 version=1.9.12
10:38:13.796 DBG build vcs.time=2025-11-20T14:52:08Z version=go1.24.0
10:38:13.797 INF [rtsp] listen addr=127.0.0.1:8554
10:38:13.797 INF [api] listen addr=127.0.0.1:1984
10:38:13.798 INF [webrtc] listen addr=:8555
[2025-12-09 10:38:13.902049] **** Consumer connecting: rtsp://127.0.0.1:8554/camera?video&audio=opus&backchannel=1
10:38:13.903 DBG [rtsp] new consumer stream=camera
10:38:14.108 DBG [ffmpeg] bin libavformat="62. 3.100" version=8.0
10:38:14.108 DBG [exec] run rtsp args=["ffmpeg","-hide_banner","-v","error","-fflags","nobuffer","-flags","low_delay","-timeout","5000000","-user_agent","go2rtc/ffmpeg","-rtsp_flags","prefer_tcp","-i","rtsp://127.0.0.1:8554/camera?audio&source=ffmpeg:camera%23audio%3Dopus","-c:a","libopus","-application:a","lowdelay","-min_comp","0","-vn","-user_agent","ffmpeg/go2rtc","-rtsp_transport","tcp","-f","rtsp","rtsp://127.0.0.1:8554/e7d6bb72e7cb25be0053aadc93618938"]
10:38:14.161 DBG [rtsp] new consumer stream=camera
10:38:14.198 DBG [streams] start producer url=rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
10:38:14.375 DBG [exec] run rtsp launch=266.930583ms
10:38:14.837 DBG [streams] start producer url=ffmpeg:camera#audio=opus
10:38:14.957 DBG [exec] [rtsp @ 0xbfac60000] RTP: PT=60: bad cseq 1544 expected=e361
I believe we are seeing two start producer lines because of the two sources for camera in the config (per your guidance earlier).
If helpful, this is what /api/streams?src=camera produces at this point:
{
"producers": [
{
"id": 3,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "192.168.99.108:554",
"url": "rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1\u0026subtype=0\u0026unicast=true\u0026proto=Onvif",
"sdp": "v=0\r\no=- 2251993276 2251993276 IN IP4 0.0.0.0\r\ns=Media Server\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=control:*\r\na=packetization-supported:DH\r\na=rtppayload-supported:DH\r\na=range:npt=now-\r\nm=video 0 RTP/AVP 96\r\na=control:trackID=0\r\na=framerate:30.000000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D4032;sprop-parameter-sets=Z01AMqaAKAC1kAA=,aO48gAA=\r\na=recvonly\r\nm=audio 0 RTP/AVP 97\r\na=control:trackID=1\r\na=rtpmap:97 MPEG4-GENERIC/16000\r\na=fmtp:97 streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408\r\na=recvonly\r\nm=application 0 RTP/AVP 107\r\na=control:trackID=4\r\na=rtpmap:107 vnd.onvif.metadata/90000\r\na=recvonly\r\nm=audio 0 RTP/AVP 111 110 109 108 107 0 106 105 8 104 103 102 101 100 97\r\na=control:trackID=5\r\na=rtpmap:111 G726-40/48000\r\na=rtpmap:110 G726-40/16000\r\na=rtpmap:109 G726-40/8000\r\na=rtpmap:108 PCMU/48000\r\na=rtpmap:107 PCMU/16000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:106 PCMA/48000\r\na=rtpmap:105 PCMA/16000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:104 L16/48000\r\na=rtpmap:103 L16/16000\r\na=rtpmap:102 L16/8000\r\na=rtpmap:101 MPEG4-GENERIC/8000\r\na=rtpmap:100 MPEG4-GENERIC/48000\r\na=rtpmap:97 MPEG4-GENERIC/16000\r\na=sendonly\r\n",
"user_agent": "go2rtc/1.9.12",
"medias": [
"video, recvonly, H264",
"audio, recvonly, MPEG4-GENERIC/16000",
"application, recvonly, VND.ONVIF.METADATA",
"audio, sendonly, G726-40/48000, G726-40/16000, G726-40/8000, PCMU/48000, PCMU/16000, PCMU/8000, PCMA/48000, PCMA/16000, PCMA/8000, L16/48000, L16/16000, L16/8000, MPEG4-GENERIC/8000, MPEG4-GENERIC/48000, MPEG4-GENERIC/16000"
],
"receivers": [
{
"id": 4,
"codec": {
"codec_name": "h264",
"codec_type": "video",
"level": 50,
"profile": "Main"
},
"childs": [
5
],
"bytes": 5750442,
"packets": 4190
},
{
"id": 7,
"codec": {
"codec_name": "aac",
"codec_type": "audio",
"sample_rate": 16000
},
"childs": [
8
],
"bytes": 46657,
"packets": 181
}
],
"senders": [
{
"id": 13,
"codec": {
"codec_name": "pcm_mulaw",
"codec_type": "audio",
"sample_rate": 16000
},
"parent": 12
}
],
"bytes_recv": 5849935
},
{
"id": 9,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "127.0.0.1:63671 forwarded 127.0.0.1:8554",
"source": "exec:ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://127.0.0.1:8554/camera?audio\u0026source=ffmpeg:camera%23audio%3Dopus -c:a libopus -application:a lowdelay -min_comp 0 -vn -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/e7d6bb72e7cb25be0053aadc93618938",
"url": "rtsp://127.0.0.1:8554/camera?audio\u0026source=ffmpeg:camera%23audio%3Dopus",
"sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.9.12\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat 62.3.100\r\nm=audio 0 RTP/AVP 96\r\nb=AS:64\r\na=rtpmap:96 opus/48000/2\r\na=control:streamid=0\r\n",
"user_agent": "ffmpeg/go2rtc",
"medias": [
"audio, recvonly, OPUS/48000/2"
],
"receivers": [
{
"id": 10,
"codec": {
"channels": 2,
"codec_name": "opus",
"codec_type": "audio",
"sample_rate": 48000
},
"childs": [
11
],
"bytes": 1175939,
"packets": 391004
}
],
"bytes_recv": 5868056
}
],
"consumers": [
{
"id": 6,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "127.0.0.1:63670",
"source": "ffmpeg:camera#audio=opus",
"sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.12\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 MPEG4-GENERIC/16000\r\na=fmtp:96 streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408\r\na=recvonly\r\na=control:trackID=0\r\n",
"user_agent": "go2rtc/ffmpeg",
"medias": [
"audio, sendonly, ANY"
],
"senders": [
{
"id": 8,
"codec": {
"codec_name": "aac",
"codec_type": "audio",
"sample_rate": 16000
},
"parent": 7,
"bytes": 46657,
"packets": 181
}
]
},
{
"id": 2,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "127.0.0.1:63668",
"sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.12\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D4032;sprop-parameter-sets=Z01AMqaAKAC1kAA=,aO48gAA=\r\na=recvonly\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 OPUS/48000/2\r\na=recvonly\r\na=control:trackID=1\r\nm=audio 0 RTP/AVP 0\r\na=rtpmap:0 PCMU/16000\r\na=sendonly\r\na=control:trackID=2\r\n",
"user_agent": "Lavf62.5.100",
"medias": [
"video, sendonly, ANY",
"audio, sendonly, OPUS",
"audio, recvonly, OPUS/48000/2, L16/16000, PCMA/16000, PCMU/16000, L16/8000, PCMA/8000, PCMU/8000"
],
"receivers": [
{
"id": 12,
"codec": {
"codec_name": "pcm_mulaw",
"codec_type": "audio",
"sample_rate": 16000
},
"childs": [
13
]
}
],
"senders": [
{
"id": 5,
"codec": {
"codec_name": "h264",
"codec_type": "video",
"level": 50,
"profile": "Main"
},
"parent": 4,
"bytes": 5750442,
"packets": 4190
},
{
"id": 11,
"codec": {
"channels": 2,
"codec_name": "opus",
"codec_type": "audio",
"sample_rate": 48000
},
"parent": 10,
"bytes": 1175942,
"packets": 391005
}
]
}
]
}
I think its the same issue as described here: https://github.com/AlexxIT/go2rtc/pull/1959
i think the backchannel=1 is also passed to ffmpeg
Can you try my fork of go2rtc pls: https://github.com/seydx/go2rtc/releases/tag/v1.9.12-cui.12
config:
streams:
camera:
- rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif#noMix
- ffmpeg:camera#audio=opus#noVideo#noBackchannel#requirePrevAudio
@seydx - unfortunately I'm having the same issue with your fork. What seems to happen is that, when a new stream connects, existing streams blow up - possibly because go2rtc is doing a reconnect on the producer, and as a result the RTP timestamp base on existing streams changes. Here's the new config:
{
"log": {
"format": "text",
"level": "debug"
},
"api": {
"listen": "127.0.0.1:1984",
"origin": "*"
},
"rtsp": {
"listen": "127.0.0.1:8554"
},
"streams": {
"camera": [
"rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif#noMix",
"ffmpeg:camera#audio=opus#noVideo#noBackchannel#requirePrevAudio"
]
}
}
For the 'NVR' connection, I've tried using URLs of rtsp://127.0.0.1:8554/camera, rtsp://127.0.0.1:8554/camera?video&audio=opus, rtsp://127.0.0.1:8554/camera?video&audio=opus&backchannel=0, rtsp://127.0.0.1:8554/camera?video&audio=opus&backchannel=1 with the same results.
Here's the start-up log, when consumer 2 connects initially (NVR) all is OK. When consumer 1 (WebRTC) connects we get the bad cseq error:
12:52:51.100 INF go2rtc platform=darwin/arm64 revision=mod.abb0684 version=1.9.12-cui.12
12:52:51.100 DBG build vcs.time=2025-11-26T19:25:04Z version=go1.24.0
12:52:51.100 INF [rtsp] listen addr=127.0.0.1:8554
12:52:51.100 INF [api] listen addr=127.0.0.1:1984
12:52:51.101 INF [webrtc] listen addr=:8555
[2025-12-09 12:52:51.205525] **** Consumer 2 connecting: rtsp://127.0.0.1:8554/camera?video&audio=opus
12:52:51.207 DBG [rtsp] new consumer stream=camera
12:52:51.406 DBG [ffmpeg] bin libavformat="62. 3.100" version=8.0
12:52:51.406 DBG [exec] run rtsp args=["ffmpeg","-hide_banner","-v","error","-fflags","nobuffer","-flags","low_delay","-timeout","5000000","-user_agent","go2rtc/ffmpeg","-rtsp_flags","prefer_tcp","-i","rtsp://127.0.0.1:8554/camera?audio&source=ffmpeg:camera%23audio%3Dopus","-c:a","libopus","-application:a","lowdelay","-min_comp","0","-vn","-user_agent","ffmpeg/go2rtc","-rtsp_transport","tcp","-f","rtsp","rtsp://127.0.0.1:8554/e7d6bb72e7cb25be0053aadc93618938"]
12:52:51.457 DBG [rtsp] new consumer stream=camera
12:52:51.483 DBG [streams] start producer url=rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
12:52:51.674 DBG [exec] run rtsp launch=268.36325ms
12:52:51.674 DBG [streams] start producer url=ffmpeg:camera#audio=opus
[2025-12-09 12:52:52.017493] **** Consumer 1 connecting: rtsp://127.0.0.1:8554/camera?video&audio=opus
12:52:52.642 DBG [exec] [rtsp @ 0x857400000] RTP: PT=60: bad cseq bfda expected=2750
Then later, if I force consumer 1 to reconnect (refreshing the browser), this causes the consumer 2 stream to blow up - it doesn't disconnect from the consumer's perspective, but it seems like the consumer sees a timestamp discontinuity (in addition to the RTP sequence number discontinuity):
[2025-12-09 12:54:35.556160] **** Consumer 1 connecting: rtsp://127.0.0.1:8554/camera?video&audio=opus
12:54:36.072 DBG [exec] [rtsp @ 0x857400000] RTP: PT=60: bad cseq 030d expected=63c6
12:54:37.289 DBG [exec] [auto_aresample_0 @ 0x856850cc0] [SWR @ 0x8578b4000] Failed to compensate for timestamp delta of 37035.004313
Edit to add: consumer 2 is not requesting a backchannel. Consumer 2 is a separate ffmpeg process and I've verified that when it's probing the RTSP input it's only seeing the H264 video track and the Opus audio (not the PCMU backchannel).
@seydx - I have done a little more digging. I am almost certain that you are right, that the new consumer connecting is triggering a reconnect (I see the RTSP TEARDOWN on the source when I wouldn't expect one) because there's something going wrong with go2rtc's producer/consumer matching. But I'm not sure why that is failing. Would it be helpful for me to post the trace log (showing the cons=xxx prod=xxx matching output) using your fork when a new consumer connects?
@seydx - I have done a little more digging. I am almost certain that you are right, that the new consumer connecting is triggering a reconnect (I see the RTSP TEARDOWN on the source when I wouldn't expect one) because there's something going wrong with go2rtc's producer/consumer matching. But I'm not sure why that is failing. Would it be helpful for me to post the trace log (showing the cons=xxx prod=xxx matching output) using your fork when a new consumer connects?
yes indeed, that would be very helpful
I think its the same issue as described here: #1959
i think the
backchannel=1is also passed to ffmpeg
No quite, that PR from myself aims at allowing backchannel=X in onvif sources, since right now it only seems to be available with rtsp and maybe ffmpeg
@seydx - first of all I wanted to say how appreciative I am of your help here. I've got 3 sets of logs for you. (Config used for each is at the top of the log.) But I think, separately, there's also a regression in your fork in the backchannel handling (see below).
In each of the logs, initially the 'NVR' stream starts, then mid-way through the log a WebRTC client connects, and things blow up (starting with a TEARDOWN of the source stream, followed by the 'bad cseq' errors).
Log 1.txt uses the 'regular' go2rtc, with my original stream configuration of ["rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif", "ffmpeg:camera#video=copy#audio=opus" ]. WebRTC client joins at line 299, after which the 'NVR' stream blows up. Note everything works on the WebRTC side, including the backchannel.
Log 2.txt is identical to log 1 but uses your fork of go2rtc. Same stream configuration. WebRTC client joins at line 305, after which the 'NVR' stream blows up. However, with this one, the WebRTC backchannel does not work. chrome://webrtc-internals is showing the correct track configuration (i.e. OPUS for audio, PCMU for backchannel), but it seems go2rtc is not passing the backchannel audio onto the producer.
Log 3.txt uses your fork of go2rtc with your revised stream configuration of ["rtsp://xxxx:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif#noMix", "ffmpeg:camera#audio=opus#noVideo#noBackchannel#requirePrevAudio"]. WebRTC client joins at line 298, and then things blow up in the same way as before.
Note that for these logs, the 'NVR' stream uses a URL of the form rtsp://127.0.0.1:8554/camera. I have also tried various combinations of query parameters (?video&audio=opus&backchannel=0, ?video&audio=opus&backchannel=1) but this doesn't seem to make any difference. I'm happy to create new logs with those URLs if that is helpful.
Separately from this, I'm suspicious that this form of stream config leads to WebRTC audio/video desync. Here, the original RTSP stream is providing video, AAC audio and PCMU backchannel. The ffmpeg stream is then providing OPUS transcoded audio. So when the WebRTC consumer requests video & OPUS & PCMU, it's getting video from the 1st stream and audio from the 2nd and the two aren't synced. So I suspect that ideally, I would want one RTSP stream providing backchannel only, and a 2nd ffmpeg stream providing video & OPUS. But I'm not sure if this is possible.
Thanks for the detailed logs
So I dug through all three logs and found the same pattern in each one. When the NVR stream is running and then a WebRTC client connects with backchannel support, go2rtc has to reconnect to the camera. The reason is that the backchannel track (trackID=5) wasn't set up initially - it only gets added when someone actually needs it.
The problem is that RTSP doesn't allow adding new tracks while the stream is playing. So go2rtc has to tear down the whole session and start fresh. You can see this happening in the logs:
09:00:06.787 TRC [streams] match cons=0 => prod=0
09:00:06.787 TRC [rtsp] client msg: RTSP reconnect
09:00:06.787 TRC [rtsp] client request: TEARDOWN rtsp://192.168.99.108...
And then right after that, the "bad cseq" errors show up because the NVR is still expecting the old sequence numbers but gets new ones from the fresh session.
Your camera does support backchannel - I can see it in the SDP response (the sendonly audio on trackID=5). But since the first consumer (NVR) doesn't need backchannel, go2rtc doesn't set it up. Then when WebRTC comes along wanting backchannel, the reconnect happens.
The #noMix flag unfortunately doesn't help here - that's only for mixing audio when multiple backchannel consumers connect at the same time. It doesn't change the reconnect behavior.
About the backchannel not working in Log 2 - the WebRTC answer actually looks correct, it shows sendrecv and PCMU/8000 for backchannel. If audio isn't getting through to the camera, that might be a separate issue. Would need to see what happens when you actually try to use the backchannel.
And yeah, your suspicion about audio/video desync with the ffmpeg transcoding setup is valid. When video comes directly from the camera and audio comes from ffmpeg with its own timing, they can drift apart. That's just a limitation of splitting the sources like that.
One thing you could try in the meantime - have you experimented with preloading the stream? Something like:
preload:
camera: video&audioµphone
This should keep the connection to the camera alive with all tracks (including backchannel) already set up. So when the WebRTC client joins, there's no need for a reconnect since everything is already there.
Is there any way you could share access to the camera? Would make it a lot easier for me to dig into this properly.
@seydx - unfortunately adding the preload section makes things worse! Now I get the reconnect / 'bad cseq' issue when the first stream (the 'NVR') connects, in addition to when the subsequent WebRTC stream connects. It looks like what is happening is that the streams are all starting as expected when go2rtc starts up (thanks to the preload), but then when the actual 'NVR' stream connects for some reason the preload streams are torn down. See attached Log 4.txt. Here, we start the first stream on line 77, then go2rtc goes through its motions including the unexpected TEARDOWN. Then on line 460 we start the WebRTC stream, and then there's another unexpected TEARDOWN.
Regarding the 'backchannel not working' issue in my previous Log 2.txt, sorry if I wasn't clear here. The only difference between Log 2 and Log 1 is that Log 2 (backchannel works) is using your fork, whereas Log 1 (backchannel doesn't work) is using the regular go2rtc. I don't think this is related to the 'unexpected TEARDOWN' issue, it's something separate and it looks like a regression in backchannel handling in your fork (since use of your fork is the only difference between the two cases). The PCMU backchannel track looks correct as far as the WebRTC client is concerned, and data is flowing into it. But it seems that never actually gets sent to the camera in your fork, whereas it does in the regular go2rtc. So I don't think this is camera specific. Have you checked / are you able to check that your fork works OK for you (backchannel via WebRTC) with streams set up similar to my config?
The camera is an Amcrest IP4M-1041W. Again I'm not sure either issue has anything to do with the camera, but I am very happy to send you one of these cameras if that would be helpful. I'll reach out to you on Discord.