packages icon indicating copy to clipboard operation
packages copied to clipboard

prometheus-node-exporter-ucode: add a plugin for go2rtc metrics

Open vooon opened this issue 2 months ago • 13 comments

📦 Package Details

Maintainer: @dhewg

Description:

  • Add a plugin to export go2rtc metrics

Requires additional api_url option to enable:

uci set prometheus-node-exporter-ucode.go2rtc=collector
uci set prometheus-node-exporter-ucode.go2rtc.api_url='http://127.0.0.1:1984'

🧪 Run Testing Details

  • OpenWrt Version: SNAPSHOT
  • OpenWrt Target/Subtarget: x86/64
  • OpenWrt Device: KVM

✅ Formalities

  • [x] I have reviewed the CONTRIBUTING.md file for detailed contributing guidelines.

vooon avatar Nov 15 '25 09:11 vooon

I'm not a go2rtc user, so I can't comment on the metrics.

But generally speaking: I would avoid relying on external processes if possible, so what about using ucode-mod-uclient instead of uclient-fetch?

Both are part of the uclient, but the former let's you use it from ucode itself.

Example: https://git.openwrt.org/?p=project/uclient.git;a=blob;f=uclient-test.uc;h=7333aa4ef2370fa199a3d67f959f16f78a212b1c;hb=HEAD

It'd be a couple more lines of code, but avoids problems like your comment states

dhewg avatar Nov 15 '25 09:11 dhewg

@dezhwrt at first i've tried to use native uclient, but cannot make it work. I'm not familiar how uhttp runs ucode, perhaps trouble was somewhere nearby uloop, don't remember.

vooon avatar Nov 15 '25 10:11 vooon

Attached patch works for me, that's a rough quick test which seems to work.

No idea if the uloop stuff is okay, it's been too long...

dhewg avatar Nov 15 '25 15:11 dhewg

github still fails it, let's try again... 0001-prometheus-node-exporter-ucode-fetch-test.patch

dhewg avatar Nov 15 '25 15:11 dhewg

@dhewg thanks! I've changed fetch using your guide, works well.

vooon avatar Nov 16 '25 11:11 vooon

Can you paste the generated metrics please, so we get an idea what's getting generated here?

Is the runtime of this collector acceptable (node_scrape_collector_duration_seconds)? I guess it takes way longer because it needs to access a http(s) resource.

As go2rtc has its own http server, it would make way more sense to let it expose its metrics itself, like you requested here: https://github.com/AlexxIT/go2rtc/issues/1388

Please follow up there, as this PR can only be a suboptimal stop gap solution. But as long as the above issue isn't resolved, this PR is okay with me.

dhewg avatar Nov 16 '25 12:11 dhewg

Here is example (after grep go2rtc):

go2rtc_up{url="http://127.0.0.1:1984"} 1
# TYPE go2rtc_producer_info gauge
go2rtc_producer_info{stream="cam20_main",format_name="",protocol="",remote_addr="",user_agent=""} 0
go2rtc_producer_info{stream="cam20_subs",format_name="rtsp",protocol="rtsp+tcp",remote_addr="192.168.25.20:554",user_agent="go2rtc/1.9.11"} 1
# TYPE go2rtc_producer_received_bytes_total counter
go2rtc_producer_received_bytes_total{stream="cam20_subs",remote_addr="192.168.25.20:554"} 75876
# TYPE go2rtc_consumer_info gauge
go2rtc_consumer_info{stream="cam20_subs",format_name="webrtc",protocol="ws+udp",remote_addr="192.168.106.1:53139 prflx",user_agent="Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0"} 1
# TYPE go2rtc_consumer_sent_bytes_total counter
go2rtc_consumer_sent_bytes_total{stream="cam20_subs",remote_addr="192.168.106.1:53139 prflx"} 75853
go2rtc_consumer_info{stream="cam20_subs",format_name="webrtc",protocol="ws+udp",remote_addr="192.168.106.1:10056 prflx",user_agent="Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0"} 1
go2rtc_consumer_sent_bytes_total{stream="cam20_subs",remote_addr="192.168.106.1:10056 prflx"} 37496
go2rtc_producer_info{stream="cam21_main",format_name="",protocol="",remote_addr="",user_agent=""} 0
go2rtc_producer_info{stream="cam21_subs",format_name="rtsp",protocol="rtsp+tcp",remote_addr="192.168.25.21:554",user_agent="go2rtc/1.9.11"} 1
go2rtc_producer_received_bytes_total{stream="cam21_subs",remote_addr="192.168.25.21:554"} 43324
go2rtc_consumer_info{stream="cam21_subs",format_name="webrtc",protocol="ws+udp",remote_addr="192.168.106.1:9141 prflx",user_agent="Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0"} 1
go2rtc_consumer_sent_bytes_total{stream="cam21_subs",remote_addr="192.168.106.1:9141 prflx"} 43333
...
go2rtc_producer_info{stream="cam33_main",format_name="",protocol="",remote_addr="",user_agent=""} 0
go2rtc_producer_info{stream="cam33_subs",format_name="rtsp",protocol="rtsp+tcp",remote_addr="192.168.25.33:554",user_agent="go2rtc/1.9.11"} 1
go2rtc_producer_received_bytes_total{stream="cam33_subs",remote_addr="192.168.25.33:554"} 19952
go2rtc_consumer_info{stream="cam33_subs",format_name="webrtc",protocol="ws+udp",remote_addr="192.168.106.1:1065 prflx",user_agent="Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0"} 1
go2rtc_consumer_sent_bytes_total{stream="cam33_subs",remote_addr="192.168.106.1:1065 prflx"} 19950
...
go2rtc_producer_info{stream="cam42_main",format_name="",protocol="",remote_addr="",user_agent=""} 0
go2rtc_producer_info{stream="cam42_subs",format_name="",protocol="",remote_addr="",user_agent=""} 0
go2rtc_producer_info{stream="cam43_main",format_name="",protocol="",remote_addr="",user_agent=""} 0
go2rtc_producer_info{stream="cam43_subs",format_name="",protocol="",remote_addr="",user_agent=""} 0
...
go2rtc_producer_info{stream="vto_main",format_name="",protocol="",remote_addr="",user_agent=""} 0
go2rtc_producer_info{stream="vto_subs",format_name="rtsp",protocol="rtsp+tcp",remote_addr="192.168.25.24:554",user_agent="go2rtc/1.9.11"} 1
go2rtc_producer_received_bytes_total{stream="vto_subs",remote_addr="192.168.25.24:554"} 1592034
go2rtc_consumer_info{stream="vto_subs",format_name="webrtc",protocol="ws+udp",remote_addr="192.168.106.1:55321 prflx",user_agent="Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0"} 1
go2rtc_consumer_sent_bytes_total{stream="vto_subs",remote_addr="192.168.106.1:55321 prflx"} 1516326
node_scrape_collector_duration_seconds{collector="go2rtc"} 0.007565396
node_scrape_collector_success{collector="go2rtc"} 1
100 54247    0 54247    0     0   147k      0 --:--:-- --:--:-- --:--:--  147k
* Connection #0 to host 192.168.60.148 left intact
node_uname_info{sysname="Linux",nodename="go2rtc",release="6.12.57",version="#0 SMP Sun Nov 16 10:23:44 2025",machine="",domainname="(none)"} 1

Currently i only check if Frigate remote connected and consumes data. With native instrumentation probably more things may be exposed... But that's enough to me.

vooon avatar Nov 16 '25 13:11 vooon

That looks rather okay at first glance. And surprisingly fast too, how fast are the others on the same host? grep for "node_scrape_collector_duration_seconds"

dhewg avatar Nov 16 '25 13:11 dhewg

Screenshot 2025-11-16 at 14-39-17 Explore - VictoriaMetrics (Prom) - Grafana

Funny, now it 2 times faster than netstat. Obviously ubus call introduced lots of past ~40ms.

vooon avatar Nov 16 '25 13:11 vooon

Screenshot 2025-11-16 at 14-44-03 Explore - VictoriaMetrics (Prom) - Grafana

How it looks now.

vooon avatar Nov 16 '25 13:11 vooon

Okay, nice.

But why switch to log?

The basic warn() does exactly what we want, no matter if its running from cmdline or procd, please use that instead.

dhewg avatar Nov 17 '25 05:11 dhewg

@dhewg done.

vooon avatar Nov 17 '25 09:11 vooon

Wtf with that CI? RV64, PPC always fails, even here, where no compilation at all...

vooon avatar Nov 25 '25 15:11 vooon