go2rtc
go2rtc copied to clipboard
AlexxIT
Home Assistant streams never shutdown after go2rtc 1.9.0 upgrade
Hi @AlexxIT & @skrashevich,
After upgrading the go2rtc addon in Home Assistant from 1.8.5 to 1.9.0, it seems like, once you view the stream, the stream never deactivates. Previously, once you stop actively viewing, the stream would shutdown in ~10 seconds, now it will stay active for hours. Downgrading back to v1.8.5 returns to the original behavior with no other changes.
This is very critical for my use case as many of the devices people use are battery powered and keeping the stream alive will completely kill the battery in a matter of hours. Is there something obvious that I am missing? I'm quite surprised just changing the addon does this as I would have assumed it was the frontend that controlled startup/shutdown, but it's 100% reproducible across two instances.
Please provide more info. What type of source. What lovelace card. What viewing technology.
After the upgrade from 1.8.5 to 1.9.0 I noticed my HA docker instance consuming much more CPU than usual. Didn't debug fully, but reverting to 1.8.5 fixes the problem for me.
It's only a reolink doorbell and an internal camera, both rtsp and doorbell with ffmpeg, I use webrtc-camera in lovelace.
What do I need to provide to debug the issue?
Please provide more info. What type of source. What lovelace card. What viewing technology.
Sorry, should have known better, Home Assistant setup is as follows:
- Generic RTSP Camera (source is ring-mqtt, which uses go2rtc with exec source to start/stop Ring camera streams on demand)
- Picture Glance card, for RTSP Source
- go2rtc addon v 1.9.0
- RTSPtoWebRTC integration pointed at go2rtc
This exact setup has worked flawlessly for quite a while now, and is used in basically this same way with 1000's of users of ring-mqtt, but after upgrading to v1.9.0 (no other changes) the stream will stay active forever after the first viewing.
Maybe some subtle bug in how RTSPtoWebRTC communicates with the go2rtc API? That's the only thing I could think of.
It feels like there is something more subtle happening here. Last night I was able to reproduce this behavior at will, on both my production and development HA boxes, and reverting to 1.8.5 resolved it in both cases.
This morning I tried again, but decided to try to WebRTC card instead, and that seemed to work (although an issue with audio, I think related to #1703), then went back to native picture glance card with RTSPtoWebRTC and it seems to be working correctly there too, at least on the development instance.
Unfortunately, I didn't get great logs from last night, but I'll try again tonight on the production instance and see what happens.
I am really at a loss as to what is happening here. I re-upgraded my production HA instance with go2rtc v1.9.0 and the problem was there again. I decided to try using the WebRTC card instead of the native picture glance with RTSPtoWebRTC, but the problem persisted there as well.
However, I couldn't figure out how this could possibly be an issue with go2rtc itself since I believe it is up to the frontend to actually shutdown the stream. It felt like the browser was somehow keeping the sessions open for reasons I didn't understand. I rebooted multiple times, but still the problem persisted. Eventually, I rebooted to stop all the streams and just let it sit for a few hours while I did something else, then, when I came back, tried again, and everything worked. Now, even after a reboot, and everything back to exactly the way it was before, it works straight away.
I have a full VM image backup of my development VM from prior to any of the upgrades, I'm tempted to restore that system to what it looked like two days ago, and attempt the process again to see if it reproduces.
Reverted to 1.8.5 also fixed my high CPU usage issue. (Using it with RTSPtoWebRTC and Frigate lovelace card)
Same problem after updating to 1.9.0 with HA and RTSPtoWebRTC. Go2RTC in docker container.
I still can't reproduce your problem. Maybe it is related to exec source. Show me what it looks like in the config
I am unable to reproduce this issue anymore. I even restored a backup from over a week ago and repeated the upgrade process, but it failed to reproduce, even though I was able to reproduce it 3 times during the initial upgrade. I'm thinking it might have been some strange caching issue with the frontend as, after looking through the code paths, I can't see how the backend could cause it.
The config is nothing special pretty much the standard go2rtc HA addon, with debug logging and RTSP listener disabled.
log:
level: debug
rtsp:
listen: ""
webrtc:
The logs during startup simply find all of my HASS generic RTSP cameras as it should:
4:41:44.475 INF go2rtc version 1.9.0 linux/amd64
14:41:44.482 INF [webrtc] listen addr=:8555
14:41:44.484 DBG [hass] load config url="hass:Front Garage Side Event"
14:41:44.485 DBG [hass] load config url="hass:Front Bedroom Side Event"
14:41:44.485 DBG [hass] load config url="hass:Back Garage Side Event"
14:41:44.482 INF [api] listen addr=:1984
14:41:44.485 DBG [hass] load config url="hass:Back Bedroom Side Event"
14:41:44.485 DBG [hass] load config url="hass:Front Doorbell Event"
14:41:44.485 DBG [hass] load config url="hass:Screen Porch Event"
14:41:44.485 DBG [hass] load config url="hass:Front Porch Event"
14:41:44.485 DBG [hass] load config url="hass:Back Garage Side Live"
14:41:44.485 DBG [hass] load config url="hass:Front Bedroom Side Live"
14:41:44.485 DBG [hass] load config url="hass:Screen Porch Live"
14:41:44.485 DBG [hass] load config url="hass:Front Porch Live"
14:41:44.485 DBG [hass] load config url="hass:Front Garage Side Live"
14:41:44.485 DBG [hass] load config url="hass:Back Bedroom Side Live"
14:41:44.485 DBG [hass] load config url="hass:Front Doorbell Live"
14:41:44.486 DBG [hass] load config url="hass:Back Porch Live"
14:41:44.486 DBG [hass] load config url="hass:Back Porch Event"
The only difference I could find from a log perspective, when viewing the stream both cases log the same start message:
12:26:50.358 DBG [streams] start producer url=rtsp://03cabcc9-ring-mqtt:8554/5c475e8366c8_event
But, when it wasn't working there was never a stop:
12:27:02.589 DBG [streams] stop producer url=rtsp://03cabcc9-ring-mqtt:8554/5c475e8366c8_event
At this point I'm willing to close this as I can reproduce it, but I think there is clearly something that happens during the upgrade process.
Fixed! https://github.com/AlexxIT/go2rtc/releases/tag/v1.9.1
Problem was with dependency pion/ice/v2 v2.3.19 for users who set fixed WebRTC UDP port. Example listen: ":8555".
By default only TCP port fixed listen: ":8555/tcp". So it took me a long time to find the problem. I'm using default settings.
Some of this updates fix the situation:
- https://github.com/pion/ice/releases/tag/v2.3.20
- https://github.com/pion/ice/releases/tag/v2.3.22