luci icon indicating copy to clipboard operation
luci copied to clipboard

luci-app-statistics: CPU plugin does not display CPU information from FreeBSD agent

Open neveripe opened this issue 4 months ago • 7 comments

Is there an existing issue for this?

  • [x] I have searched among all existing issues (including closed issues)

screenshots or captures

Image Image Image Image

Actual behaviour

  1. I have OpenWRT 24.10.4 running collectd, which collects data from several other hosts. 3 of them are also OpenWRT, 1 of them is FreeBSD server.

  2. CPU graphs and Memory graphs are not being displayed in the UI (I'm getting 4 empty images), and when debugging through browser through "Networking" tab - it turns out that size of the content served for these pictures (graphs) is 0 bytes.

  3. Two other graphs - Load (load averages) and networking work perfectly and served with normal size.

  4. Corresponding RRD files are present on the OpenWRT instance and being updated and seem to contain data:

RRD which doe NOT generate graph:

root@OpenWrt:/mnt/rrd# rrdtool info php71/cpu-0/percent-user.rrd
filename = "php71/cpu-0/percent-user.rrd"
rrd_version = "0001"
step = 10
last_update = 1761418875
ds[value].type = "GAUGE"
ds[value].minimal_heartbeat = 20
ds[value].min = 0.0000000000e+00
ds[value].max = 1.0010000000e+02
ds[value].last_ds = "UNKN"
ds[value].value = 7.1735395189e+01
ds[value].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 360
rra[0].pdp_per_row = 1
rra[0].xff = 1.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 288
rra[1].pdp_per_row = 30
rra[1].xff = 1.0000000000e-01
rra[1].cdp_prep[0].value = 4.3281827450e+01
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 288
rra[2].pdp_per_row = 210
rra[2].xff = 1.0000000000e-01
rra[2].cdp_prep[0].value = 2.0496579446e+03
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 288
rra[3].pdp_per_row = 930
rra[3].xff = 1.0000000000e-01
rra[3].cdp_prep[0].value = 7.6698426221e+03
rra[3].cdp_prep[0].unknown_datapoints = 136
rra[4].cf = "AVERAGE"
rra[4].rows = 288
rra[4].pdp_per_row = 10980
rra[4].xff = 1.0000000000e-01
rra[4].cdp_prep[0].value = 7.6698426221e+03
rra[4].cdp_prep[0].unknown_datapoints = 46

RRD data which DOES generate graph:

root@OpenWrt:/mnt/rrd# rrdtool info OpenWrt/cpu-0/percent-user.rrd
filename = "OpenWrt/cpu-0/percent-user.rrd"
rrd_version = "0001"
step = 30
last_update = 1761418935
ds[value].type = "GAUGE"
ds[value].minimal_heartbeat = 60
ds[value].min = 0.0000000000e+00
ds[value].max = 1.0010000000e+02
ds[value].last_ds = "UNKN"
ds[value].value = 1.3000000000e+01
ds[value].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 288
rra[0].pdp_per_row = 1
rra[0].xff = 1.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 288
rra[1].pdp_per_row = 10
rra[1].xff = 1.0000000000e-01
rra[1].cdp_prep[0].value = 6.0000000000e-01
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 288
rra[2].pdp_per_row = 70
rra[2].xff = 1.0000000000e-01
rra[2].cdp_prep[0].value = 4.8133333333e+01
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 288
rra[3].pdp_per_row = 310
rra[3].xff = 1.0000000000e-01
rra[3].cdp_prep[0].value = 1.3651666667e+02
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[4].cf = "AVERAGE"
rra[4].rows = 288
rra[4].pdp_per_row = 3660
rra[4].xff = 1.0000000000e-01
rra[4].cdp_prep[0].value = 1.2346666667e+02
rra[4].cdp_prep[0].unknown_datapoints = 0

The only difference I see is "unknown_datapoints", but I don't know what causes it, also, the other graph which is not displayed - Memory - does not have those "unknown_datapoints".

root@OpenWrt:/mnt/rrd# rrdtool info php71/memory/memory-active.rrd
filename = "php71/memory/memory-active.rrd"
rrd_version = "0001"
step = 10
last_update = 1761419095
ds[value].type = "GAUGE"
ds[value].minimal_heartbeat = 20
ds[value].min = 0.0000000000e+00
ds[value].max = 2.8147497671e+14
ds[value].last_ds = "UNKN"
ds[value].value = 2.5843712000e+08
ds[value].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 360
rra[0].pdp_per_row = 1
rra[0].xff = 1.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 288
rra[1].pdp_per_row = 30
rra[1].xff = 1.0000000000e-01
rra[1].cdp_prep[0].value = 1.7243197440e+09
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 288
rra[2].pdp_per_row = 210
rra[2].xff = 1.0000000000e-01
rra[2].cdp_prep[0].value = 8.6791598080e+10
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 288
rra[3].pdp_per_row = 930
rra[3].xff = 1.0000000000e-01
rra[3].cdp_prep[0].value = 1.4135382671e+11
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[4].cf = "AVERAGE"
rra[4].rows = 288
rra[4].pdp_per_row = 10980
rra[4].xff = 1.0000000000e-01
rra[4].cdp_prep[0].value = 1.3772051251e+11
rra[4].cdp_prep[0].unknown_datapoints = 0

Expected behaviour

I would expect graphs for all metrics to be generated.

Steps to reproduce

Report CPU and memory metrics from FreeBSD server to collectd running on OpenWRT.

Additional Information

root@OpenWrt:/mnt/rrd# ubus call system board
{
        "kernel": "6.6.110",
        "hostname": "OpenWrt",
        "system": "ARMv7 Processor rev 1 (v7l)",
        "model": "Linksys WRT32X",
        "board_name": "linksys,wrt32x",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "24.10.4",
                "revision": "r28959-29397011cc",
                "target": "mvebu/cortexa9",
                "description": "OpenWrt 24.10.4 r28959-29397011cc",
                "builddate": "1760891865"
        }
}

What browsers do you see the problem on?

Chrome

Relevant log output


neveripe avatar Oct 25 '25 19:10 neveripe

@jow- @hnyman ?

systemcrash avatar Oct 25 '25 19:10 systemcrash

Additional note - everything is okay with OpenWRT agents' reports - all of their graphs displayed perfectly fine.

neveripe avatar Oct 25 '25 20:10 neveripe

I can also provide exact RRD files if needed (privately) for investigation.

neveripe avatar Oct 25 '25 23:10 neveripe

Pure guess, but there is a difference in "step", 10 sec vs. 30 sec time period. And the non-working one also has variance in database record counts (288, 360, etc.) while the working one has uniformly the OpenWrt standard 288.

Note also that for size reasons we are using a 20-year-old rrdtool version, which has less functionality than the curren rrdtool. The is e.g. 32 but time Vs 64 bit.

Rrdtool can also be picky about database generated by different architecture.

But as you say that load graph works ok, I am not sure which of the above observations is still valid regarding the difference between working and non-working BSD data files.

hnyman avatar Oct 26 '25 04:10 hnyman

I'm happy to provide each and every piece of data/evidence and (if necessary) even provide access to both systems involved (FreeBSD server and OpenWRT router) if necessary to collect required information. I've been trying to enable any kind of debugging on uhttpd/luci side but failed... :(

neveripe avatar Oct 26 '25 04:10 neveripe

Hm. 64bit vs 32bit seems plausible (or at least reasonable), but yes, if 2 other graphs work and other two do not (coming from the same 64bit system to 32bit (I assume armv7l is 32bit?) system) this is probably no-go.

I would suspect some problem with CPU metrics in collectd then - so that actual data written to RRDs are wrong? (I guess I might be the first person to send collectd data from FreeBSD to Linux (and especially to OpenWRT)?

Anyway - happy to help to get each and every piece of data and help as much as I can to debug this.

collectd on FreeBSD side:

collectd5-5.12.0_23 Systems & network statistics collection daemon

collectd on OpenWRT side:

collectd - 5.12.0-r54

neveripe avatar Oct 26 '25 04:10 neveripe

Sent you an email with the full data, Hannu. Hope that helps...

neveripe avatar Oct 26 '25 05:10 neveripe