Allow multiple parents and fix hanging producer

[seydx]

This fixes the issue with hanging producers which occurred when, for example, a stream was opened by multiple consumers (with backchannel).

This PR extends the receiver/sender parents and allows them, similar to the children, to have multiple nodes. These are then also cleaned up when closing, which prevents the hanging producer problem.

To reproduce, open several tabs for a stream with backchannel support within ui (webrtc, video+audio+microphone) and close after. The consumer will be closed, but the producer will remain and hang

[PetePeter]

I think i have experienced this problem. let me try grab a copy of your version and see if it works for me. hopefully Alex incorporates this in the next release if it fixes it.

[seydx]

I think i have experienced this problem. let me try grab a copy of your version and see if it works for me. hopefully Alex incorporates this in the next release if it fixes it.

https://github.com/seydx/go2rtc/actions/runs/11653773039

[PetePeter]

Thanks! your version is based on 1.9.6 which has a new bug for me. I can only connect to its once. Once I disconnect my phone or webbrowser HA/webrtc card (i havent tried others) the streams never shut down. I end up with a stuck consumer on the go2rtc log which wont go away by itself, and then the webrtc card never loads again - until i restart go2rtc.

sad.

fwiw my config is

doorbell_audio2way:

• rtsp://admin:[email protected]/Preview_01_sub
• ffmpeg:doorbell_audio2way#audio=opus#audio=copy

and what works fine on 1.9.4 - except, then, sometimes, eventually, theres a very long long long long list of children, and a stream that wont work properly anymore. where how-long-eventually is maybe 10 or so activations of HA feed before it turns useless.

ha card: type: custom:webrtc-camera ui: false streams:

• url: doorbell_audio2way mode: webrtc media: video,audio,microphone name: Speaking

camera: reolink doorbell poe with second-latest fw as i dont want to update

[seydx]

Thanks! your version is based on 1.9.6 which has a new bug for me. I can only connect to its once. Once I disconnect my phone or webbrowser HA/webrtc card (i havent tried others) the streams never shut down. I end up with a stuck consumer on the go2rtc log which wont go away by itself, and then the webrtc card never loads again - until i restart go2rtc.

sad.

fwiw my config is

doorbell_audio2way:

• rtsp://admin:[email protected]/Preview_01_sub
• ffmpeg:doorbell_audio2way#audio=opus#audio=copy

and what works fine on 1.9.4 - except, then, sometimes, eventually, theres a very long long long long list of children, and a stream that wont work properly anymore. where how-long-eventually is maybe 10 or so activations of HA feed before it turns useless.

ha card: type: custom:webrtc-camera ui: false streams:

• url: doorbell_audio2way mode: webrtc media: video,audio,microphone name: Speaking

camera: reolink doorbell poe with second-latest fw as i dont want to update

Yeah there are two issues

1. It's a known issue with streams that point to themselves (like your ffmpeg source) where the “Producer” hangs when you close the stream.

2. I have not tested the changes I have made with such a stream. I will have a look

Edit: Alex knows about the 1st issue and said he is working on it

[PetePeter]

okay thanks for the update. I guess i can just make the ffmpeg a copy of the whole thing again to see if it works

[seydx]

doorbell_audio2way:

• rtsp://admin:[email protected]/Preview_01_sub
• ffmpeg:doorbell_audio2way#audio=opus#audio=copy

what about this one?

doorbell_audio2way:
  - ffmpeg:rtsp://admin:[email protected]/Preview_01_sub#video=copy#audio=copy#audio=opus

Edit: Two way will not work with this

[PetePeter]

Yep. Now i have

    doorbell_audio2way:
    - rtsp://admin:[email protected]/Preview_01_sub
    - ffmpeg:rtsp://admin:[email protected]/Preview_01_sub#audio=opus#audio=copy

and it does work with 2 way :) i gave it a whirl and it

PC (no mic) and phone 2way, 2 way worked and i closed both and all was well.

phone (with mic) and tablet (with mic) it went crazy. phone was doing the mic > doorbell tablet was doing the audio doorbell > tablet (and its stream kept on restrating) :D

and when i closed everything, this was left behind

{
    "producers": [
        {
            "id": 185,
            "format_name": "rtsp",
            "protocol": "rtsp+tcp",
            "remote_addr": "10.98.1.215:554",
            "url": "rtsp://admin:[email protected]/Preview_01_sub",
            "sdp": "v=0\r\no=- 1730559630899010 1 IN IP4 10.98.1.215\r\ns=Session streamed by \"preview\"\r\nt=0 0\r\na=tool:BC Streaming Media v202210012022.10.01\r\na=type:broadcast\r\na=control:*\r\na=range:npt=now-\r\na=x-qt-text-nam:Session streamed by \"preview\"\r\nm=video 0 RTP/AVP 96\r\nc=IN IP4 0.0.0.0\r\nb=AS:8192\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=640033;sprop-parameter-sets=Z2QAM6wVFKCgPZA=,aO48sA==\r\na=recvonly\r\na=control:track1\r\nm=audio 0 RTP/AVP 97\r\nc=IN IP4 0.0.0.0\r\nb=AS:8192\r\na=rtpmap:97 MPEG4-GENERIC/16000\r\na=fmtp:97 streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408\r\na=recvonly\r\na=control:track2\r\nm=audio 0 RTP/AVP 0\r\na=control:track3\r\na=rtpmap:0 PCMU/8000\r\na=sendonly",
            "user_agent": "go2rtc/1.9.6",
            "medias": [
                "video, recvonly, H264",
                "audio, recvonly, MPEG4-GENERIC/16000",
                "audio, sendonly, PCMU/8000"
            ],
            "receivers": [
                {
                    "id": 187,
                    "codec": {
                        "codec_name": "h264",
                        "codec_type": "video",
                        "level": 51,
                        "profile": "High"
                    },
                    "childs": [
                        132,
                        137,
                        148,
                        156,
                        164,
                        175,
                        183
                    ],
                    "bytes": 2356204,
                    "packets": 1907
                }
            ],
            "senders": [
                {
                    "id": 188,
                    "codec": {
                        "codec_name": "pcm_mulaw",
                        "codec_type": "audio",
                        "sample_rate": 8000
                    },
                    "parents": [
                        125
                    ]
                },
                {
                    "id": 189,
                    "codec": {
                        "codec_name": "pcm_mulaw",
                        "codec_type": "audio",
                        "sample_rate": 8000
                    },
                    "bytes": 98560,
                    "packets": 616
                }
            ],
            "bytes_recv": 2380296,
            "bytes_send": 108416
        },
        {
            "url": "ffmpeg:rtsp://admin:[email protected]/Preview_01_sub#audio=opus#audio=copy"
        }
    ],
    "consumers": []
}

But i could reconnect and it was ok again. I've had the above behaviour where the childs keep on growing, and eventually it all stops working on that stream (on 1.9.4)

This exact case above, is the one I had hoped your changes would fix. Because, this causes my 2way audio stream to die eventually in 1.9.4

[PetePeter]

I did a stress test with this config on my reolink doorbell:

  doorbell_audio2way:
  - rtsp://admin:[email protected]/Preview_01_sub
  - ffmpeg:doorbell_audio2way#audio=opus#audio=copy
  doorbell_liveview:
  - rtsp://admin:[email protected]/Preview_01_sub#video=copy#audio=copy
  doorbell_recording:
  - rtsp://admin:[email protected]/Preview_01_main
  doorbell_tts:
  - rtsp://admin:[email protected]/Preview_01_main
  - ffmpeg:doorbell_tts#audio=opus#audio=copy

and

this ha card

type: custom:webrtc-camera
ui: true
streams:
  - url: doorbell_liveview
    mode: webrtc
    media: video,audio
    name: Muted
  - url: doorbell_audio2way
    mode: webrtc
    media: video,audio,microphone
    name: Speaking

In my stress test i basically pressed the muted/speaking button to switch streams quickly until go2rtsp shows upto 20 consumers on each. and then watched how far down they went, followed by closing the ha browser window

go2rtc 1.9.2 made it back down to 0 1.9.3 got stuck on 2 on the audio2way and 1 on the liveview 1.9.4 i didnt try since i know it already gets stuck sometimes

not sure if my issue is the same as this issue or a separate one :/

[seydx]

@PetePeter

can you try this one pls: https://github.com/seydx/go2rtc/actions/runs/11729591864

[PetePeter]

@seydx works perfectly! stress test on two devices switching between streems like mad - resolves itself down to 0 eventually. even the self referential streams like my original config where i ffmpg itself, also works fine.

Thank you!

[felipecrs]

There's a chance this PR can also fix https://github.com/AlexxIT/go2rtc/issues/530, I believe.

[PetePeter]

Oopsie I managed to crash it again.

I had been trying out 2way audio while also doing TTS to see what would happen. and now it wont work anymore [obs a restart would fix it again]

attached are the 'info's of the 4 streams i use. The recording is still actually active as frigate is recording tts is the one i use for tts. liveview is the one i use when using the webrtc card without microphone 2wayaudio is the one i use when using the webrtc card with microphone

the last two are frozen and dead.

I think something error'd somewhere when TTS and 2way audio wanted to talk at the same time, and the 2way audio and the liveview (which is the same kind of stream as the 2wayaudio) also froze.

doorbell_audio2way.txt doorbell_liveview.txt doorbell_recording.txt doorbell_tts.txt

[seydx]

Oopsie I managed to crash it again.

I had been trying out 2way audio while also doing TTS to see what would happen. and now it wont work anymore [obs a restart would fix it again]

attached are the 'info's of the 4 streams i use. The recording is still actually active as frigate is recording tts is the one i use for tts. liveview is the one i use when using the webrtc card without microphone 2wayaudio is the one i use when using the webrtc card with microphone

the last two are frozen and dead.

I think something error'd somewhere when TTS and 2way audio wanted to talk at the same time, and the 2way audio and the liveview (which is the same kind of stream as the 2wayaudio) also froze.

doorbell_audio2way.txt doorbell_liveview.txt doorbell_recording.txt doorbell_tts.txt

can you post your config pls

[PetePeter]

doh sorry

    doorbell_audio2way:
    - rtsp://admin:[email protected]/Preview_01_sub
    - ffmpeg:doorbell_audio2way#audio=opus#audio=copy#audio=aac
    doorbell_liveview:
    - rtsp://admin:[email protected]/Preview_01_sub
    - ffmpeg:doorbell_liveview#audio=opus#audio=copy#audio=aac

    doorbell_tts:
    - rtsp://admin:[email protected]/Preview_01_main
    - ffmpeg:doorbell_tts#audio=opus#audio=copy    
    doorbell_recording:
    - rtsp://admin:[email protected]/Preview_01_main

also attached is another hanging stream. I managed to hang the 'liveview' one this time. I was doing tts on my phone as someone was arriving, while viewing the liveview stream; just as HA automation triggered also doing a tts.

the webrtc card config is

type: custom:webrtc-camera
ui: true
streams:
  - url: doorbell_liveview
    mode: webrtc
    media: video,audio
    name: Muted
  - url: doorbell_audio2way
    mode: webrtc
    media: video,audio,microphone
    name: Speaking

[2] doorbell_liveview.txt

[seydx]

@PetePeter

https://github.com/seydx/go2rtc/actions/runs/11755681506

[PetePeter]

thankyou. will check maybe tomorrow or the day after. Sunday is too busy

[PetePeter]

it was looking really good and after much testing all was well. but then

2way stream hung. same config on everything

2way.txt

[PetePeter]

Maybe add some logging so that i can report that back when i cause a hang?

[PetePeter]

fyi: I'll be away till Tue

[PetePeter]

ok i'm back if you want me to test any builds since last time.

[dagleaves]

Update from my Reolink locations. This does not appear to fix the issue with hanging audio consumers #1254 Video stream still fails to come back. Guess they're unrelated.

[AlexxIT]

According to @seydx this PR is in development and requires additional testing

[seydx]

Closing this as it seems its fixed with v1.9.9