dellhw_exporter icon indicating copy to clipboard operation
dellhw_exporter copied to clipboard

chassis_batteries unavailable

Open lewispb opened this issue 7 months ago • 8 comments

In latest BIOS versions chassis_batteries no longer reports metrics.

This is because of the following described at https://www.dell.com/support/kbdoc/en-uk/000227413/14g-intel-poweredge-coin-cell-battery-changes-in-august-2024-firmware

Removal of CMOS battery sensor: As part of implementing the logging changes above, the CMOS battery sensor has been disabled to avoid inconsistencies in health reporting. As a result, the CMOS battery no longer appears in any management interface, including: iDRAC web UI

Logs:

Jun 06 14:14:46 <server name> dellhw_exporter[1772793]: time="2025-06-06T14:14:46Z" level=error msg="chassis_batteries collector failed after 0.243436s: failed to execute command (\"/opt/dell/srvadmin/bin/omreport [chassis batteries -fmt ssv]\"). exit status 255"

lewispb avatar Sep 12 '25 09:09 lewispb

Thanks for bringing this to my attention!

Could you send the output of the following commands here so I can see if I can make the exporter "auto discover" if chassis batteries is (still) available and disable the collector if needed.

/opt/dell/srvadmin/bin/omreport chassis batteries -fmt ssv

/opt/dell/srvadmin/bin/omreport chassis batteries

galexrt avatar Sep 13 '25 09:09 galexrt

Thanks @galexrt.

lewis@kvm-01 ~ [255]> /opt/dell/srvadmin/bin/omreport chassis batteries -fmt ssv
Error! No battery probes found on this system.

lewis@kvm-01 ~ [255]> /opt/dell/srvadmin/bin/omreport chassis batteries
Error! No battery probes found on this system.

Let me know if you need anything else.

lewispb avatar Sep 13 '25 17:09 lewispb

I should have also said we were running dellhw_exporter 1.13.9 when we noticed this issue.

lewispb avatar Sep 15 '25 10:09 lewispb

I have pushed a commit that adds a flag --collectors-check it is a list of collectors, right now only chassis_batteries. The flag will cause the exporter to check if, e.g., chassis batteries would return metrics or not.

I will create a new release candidate tomorrow.

galexrt avatar Sep 22 '25 20:09 galexrt

@lewispb v2.0.0-rc.3 has been released.

Please let me know if v2.0.0-rc.3 works for you and automatically disables the chassis_battery collector (no more errors about it in the logs)

galexrt avatar Sep 24 '25 18:09 galexrt

Thanks @galexrt. So far I haven't got past our Cinc cookbook test, which runs in Dokken.

Is it --check-collectors or --collectors-check? The commit message and code conflict about that. Maybe it'd be good to clarify in the help text.

unknown flag: --collectors-check
dellhw-exporter.service: Main process exited, code=exited, status=2/INVALIDARGUMENT

or

unknown flag: --check-collectors

lewispb avatar Oct 09 '25 09:10 lewispb

@lewispb The flag is --collectors-check=... and I just realized that it doesn't appear in the help output nor has been added to the docs. That should be fixed in the latest commits (including a faq section entry.)

The flag is a "string slice" flag and to enable it, would need to be specified like this --collectors-check=chassis_batteries on the exporter (an example for setting the flag via the Helm chart has been added to the latest chart release 1.2.0.)

galexrt avatar Oct 21 '25 09:10 galexrt

Thanks for clarifying @galexrt. I finally got back to this today and upgraded our fleet - the new flag is working nicely.

One nit, there's some logs on boot:

24T15:45:48.769491249Z","level":"INFO","msg":"enabled collectors","collectors":["chassis","chassis_batteries","fans","firmwares","memory","nics","processors","ps","ps_amps_sysboard_pwr","storage_battery","storage_controller","storage_enclosure","storage_pdisk","storage_vdisk","system","temps","version","volts"]}
Nov 24 15:45:48 kvm-01 dellhw_exporter[3098788]: {"time":"2025-11-24T15:45:48.769971672Z","level":"INFO","msg":"Listening on","address":"[::]:9137"}
Nov 24 15:45:48 kvm-01 dellhw_exporter[3098788]: {"time":"2025-11-24T15:45:48.769994833Z","level":"INFO","msg":"TLS is disabled.","http2":false,"address":"[::]:9137"}

Note that chassis_batteries is in the enabled_collectors. I understand it is technically enabled and then the collectors-check disables it but it might be nice to confirm on boot that the collectors-check flag is active and working.

lewispb avatar Nov 24 '25 15:11 lewispb