AlexxIT
Audio is not recovered after some hiccup (video does)
For this specific camera, audio in go2rtc simply gets blank after some time. Consuming directly from camera is working normally, at least through VLC. I believe this may be related to some timeout. Maybe when there's some timeout (because of wi-fi), go2rtc breaks and does not recover audio.
- Camera: Inqmega IL-PTZ381-2M-C
- Connection: Wi-Fi
go2rtc.yaml
streams:
rua:
- rtsp://admin:[email protected]/0/av1#timeout=10
rua_hd:
- rtsp://admin:[email protected]/0/av0#timeout=10
I am testing it by opening rtsp://192.168.1.10:8554/rua_hd in VLC.
Show the stream probe. What if you use ffmpeg source? What's with the video? Does it keep going when the audio goes out?
The go2rtc version is 1.9.9.
I will try with ffmpeg source (ffmpeg:rtsp://admin:[email protected]/0/av0#video=copy#audio=copy).
The video works. It does not stop like the audio does.
The probe:
{
"producers": [
{
"id": 2,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "192.168.1.33:554",
"url": "rtsp://admin:[email protected]/0/av0",
"sdp": "v=0\r\no=StreamingServer 3331435948 1116907222000 IN IP4 192.168.1.33\r\ns=h264.mp4\r\ni=TAS-Tech Live Cast\r\nc=IN IP4 192.168.1.33\r\nt=0 0\r\na=recvonly\r\na=range:npt=now-\r\nm=video 0 RTP/AVP 96\r\na=control:rtsp://192.168.1.33:554/0/video0\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1\r\na=framesize:96 1280-720\r\nm=audio 0 RTP/AVP 8\r\na=control:rtsp://192.168.1.33:554/0/audio\r\na=rtpmap:8 pcma/8000\r\na=ptime:40\r\n\r\n",
"user_agent": "go2rtc/1.9.9",
"medias": [
"video, recvonly, H264",
"audio, recvonly, PCMA/8000"
],
"receivers": [
{
"id": 3,
"codec": {
"codec_name": "h264",
"codec_type": "video"
},
"childs": [
4,
38,
105
],
"bytes": 648715,
"packets": 513
},
{
"id": 5,
"codec": {
"codec_name": "pcm_alaw",
"codec_type": "audio",
"sample_rate": 8000
},
"childs": [
6,
39,
106
],
"bytes": 55680,
"packets": 174
}
],
"bytes_recv": 712639
}
],
"consumers": [
{
"id": 1,
"format_name": "webrtc",
"protocol": "ws+udp",
"remote_addr": "192.168.1.15:56032 host",
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36",
"medias": [
"video, sendonly, VP8, VP9, H264, AV1, H265",
"audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, L16, PCML"
],
"senders": [
{
"id": 4,
"codec": {
"codec_name": "h264",
"codec_type": "video"
},
"parent": 3,
"bytes": 648715,
"packets": 513
},
{
"id": 6,
"codec": {
"codec_name": "pcm_alaw",
"codec_type": "audio",
"sample_rate": 8000
},
"parent": 5,
"bytes": 55680,
"packets": 174
}
],
"bytes_send": 711907
},
{
"id": 37,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "127.0.0.1:53614",
"sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.9\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1\r\na=recvonly\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 PCMA/8000\r\na=recvonly\r\na=control:trackID=1\r\n",
"user_agent": "FFmpeg Frigate/0.16.0-a0a5aad",
"medias": [
"video, sendonly, ANY",
"audio, sendonly, ANY"
],
"senders": [
{
"id": 38,
"codec": {
"codec_name": "h264",
"codec_type": "video"
},
"parent": 3,
"bytes": 365864,
"packets": 295
},
{
"id": 39,
"codec": {
"codec_name": "pcm_alaw",
"codec_type": "audio",
"sample_rate": 8000
},
"parent": 5,
"bytes": 39360,
"packets": 123
}
],
"bytes_send": 411912
},
{
"id": 104,
"format_name": "probe",
"protocol": "http",
"remote_addr": "172.30.33.5:33846 forwarded 192.168.1.15",
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36",
"medias": [
"video, sendonly, ALL",
"audio, sendonly, ALL",
"audio, recvonly, ANY"
],
"senders": [
{
"id": 105,
"codec": {
"codec_name": "h264",
"codec_type": "video"
},
"parent": 3
},
{
"id": 106,
"codec": {
"codec_name": "pcm_alaw",
"codec_type": "audio",
"sample_rate": 8000
},
"parent": 5
}
]
}
]
}
I don't know what problems there might be with the PCMA codec. This is the simplest single-byte codec without any headers.
The issue happens even when using ffmpeg as source.
When using ffmpeg, I caught this in my logs:
1:58:02.159 PM | warn | undefined error=read tcp 127.0.0.1:8554->127.0.0.1:59024: i/o timeout url=ffmpeg:rtsp://admin:[email protected]/0/av0#video=copy#audio=copy caller=github.com/AlexxIT/go2rtc/internal/streams/producer.go:170
-- | -- | --
2:00:20.565 PM | warn | undefined error=read tcp 127.0.0.1:8554->127.0.0.1:36126: i/o timeout url=ffmpeg:rtsp://admin:[email protected]/0/av0#video=copy#audio=copy caller=github.com/AlexxIT/go2rtc/internal/streams/producer.go:170
2:01:21.203 PM | warn | undefined error=read tcp 127.0.0.1:8554->127.0.0.1:47350: i/o timeout url=ffmpeg:rtsp://admin:[email protected]/0/av0#video=copy#audio=copy caller=github.com/AlexxIT/go2rtc/internal/streams/producer.go:170
2:02:05.910 PM | error | [exec] timeout source=exec:/usr/lib/ffmpeg/5.0/bin/ffmpeg -hide_banner -v error -allowed_media_types video+audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://admin:[email protected]/0/av0 -c:v copy -c:a copy -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/58886085a87341dcb46ef221fb9e67f0
2:02:05.910 PM | warn | [rtsp] error=streams: exec: timeout stream=rua_hd
I think audio stopped after this. Video was kept going.
Check stream info after audio stopped. If bytes for audio track changes for producer and for consumer.
@AlexxIT just one correction, I ran a few more tests and I believe it's working with ffmpeg source. The audio does not break when using ffmpeg source.
Check stream info after audio stopped. If bytes for audio track changes for producer and for consumer.
The bytes for the video stream keep increasing, but when the audio stream breaks, then the audio bytes stop increasing.
In my example below I had "bytes": 174400, forever.
Also, I can catch the issue in real time. Normally it takes less than a minute to reproduce the issue after restarting go2rtc.
If you want, I can send you direct access link to the camera stream so that you can reproduce it by yourself.
{
"producers": [
{
"id": 22,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "192.168.1.33:554",
"url": "rtsp://admin:[email protected]/0/av0",
"sdp": "v=0\r\no=StreamingServer 3331435948 1116907222000 IN IP4 192.168.1.33\r\ns=h264.mp4\r\ni=TAS-Tech Live Cast\r\nc=IN IP4 192.168.1.33\r\nt=0 0\r\na=recvonly\r\na=range:npt=now-\r\nm=video 0 RTP/AVP 96\r\na=control:rtsp://192.168.1.33:554/0/video0\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1\r\na=framesize:96 1280-720\r\nm=audio 0 RTP/AVP 8\r\na=control:rtsp://192.168.1.33:554/0/audio\r\na=rtpmap:8 pcma/8000\r\na=ptime:40\r\n\r\n",
"user_agent": "go2rtc/1.9.9",
"medias": [
"video, recvonly, H264",
"audio, recvonly, PCMA/8000"
],
"receivers": [
{
"id": 23,
"codec": {
"codec_name": "h264",
"codec_type": "video"
},
"childs": [
24,
64
],
"bytes": 6075438,
"packets": 4703
},
{
"id": 25,
"codec": {
"codec_name": "pcm_alaw",
"codec_type": "audio",
"sample_rate": 8000
},
"childs": [
26,
65
],
"bytes": 174400,
"packets": 545
}
],
"bytes_recv": 6312814
}
],
}
I won't be able to check it for the next two weeks. I can look at this stream later.
Just FYI, checked again against master (aa0ece2d1e396cd4b044f2e069d4ec394586ce77) and this issue is still happening.
I was able to capture the problem:
https://github.com/user-attachments/assets/803a31f2-d977-4969-9c86-18f477de6a62
You can check at 0:14 that RTSP wrong input is logged, audio stops producing, and the video gets distorted for some moments. The video recovers, but audio never recovers.
In VLC, audio "never" breaks, and video doesn't get distorted either.
Updates:
@AlexxIT was not able to reproduce the issue himself with direct access to the camera. We suspect it's due to other network issues. Also because most tunnel services are blocked in Russia, like ngrok and pinggy. I found one that works in Russia though: https://xtunnel.ru.
@seydx was able to reproduce the issue and he investigated it. He found out that:
- Most likely the camera is having problems interleaving TCP packets. Maybe poor wi-fi performance.
rtsp://+#transport=udpworks around it- It works in VLC because VLC utilizes UDP by default
Still unsure why it works with ffmpeg:.
Ok, I tested this better, and I can confirm this issue also happens with ffmpeg: source. It just seem a bit harder to reproduce it for some reason.
What if we apply the timeout logic when one of the tracks stop increasing bytes but the other continues?
I.e. if video bytes are growing but audio bytes are not, provoke a reconnect if the audio bytes don't grow for 5 seconds (or whatever the timeout user set).
Is it a good idea? Not sure. Could the bytes stop growing for reasons that should not cause a reconnect?