node_exporter
node_exporter copied to clipboard
[RFC] Add perf MSR collection
This change adds support for reading from model specific registers (MSRs) that are CPU/architecture dependent.
An example using a MSR from the AMD processor handbook:
sudo ./node_exporter --collector.perf --collector.perf.msr test:0x00E8:8:big
ts=2023-01-14T20:00:03.446Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=, branch=, revision=)"
ts=2023-01-14T20:00:03.446Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.4, user=, date=)"
ts=2023-01-14T20:00:03.446Z caller=node_exporter.go:183 level=warn msg="Node Exporter is running as root user. This exporter is designed to run as unprivileged user, root is not required."
&{offset:232 name:test order:big msr:0xc0000143d0 buf:[0 0 0 0 0 0 0 0]}
ts=2023-01-14T20:00:03.734Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.
+)($|/)
ts=2023-01-14T20:00:03.735Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|
iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$
ts=2023-01-14T20:00:03.735Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$
ts=2023-01-14T20:00:03.736Z caller=node_exporter.go:110 level=info msg="Enabled collectors"
Run a collection:
$ curl localhost:9100/metrics | grep msr
# HELP node_msr_test MSR: name test offset: 232 size: 8
# TYPE node_msr_test counter
node_msr_test{cpu="0"} 6.04490478872481e+18
node_msr_test{cpu="1"} 1.4632347443669238e+19
node_msr_test{cpu="10"} 1.3530478115507143e+19
node_msr_test{cpu="11"} 1.6970024784958063e+19
node_msr_test{cpu="12"} 6.414291144099758e+17
node_msr_test{cpu="13"} 1.787462845160489e+19
node_msr_test{cpu="14"} 7.104828714084467e+18
node_msr_test{cpu="15"} 1.4859343131320517e+19
node_msr_test{cpu="2"} 2.1483661629985915e+18
node_msr_test{cpu="3"} 9.60002997629223e+18
node_msr_test{cpu="4"} 1.21383391944434e+19
node_msr_test{cpu="5"} 1.6806281566140498e+19
node_msr_test{cpu="6"} 1.0722963340868452e+19
node_msr_test{cpu="7"} 6.698654098914279e+18
node_msr_test{cpu="8"} 1.2851969305356534e+18
node_msr_test{cpu="9"} 1.0905766915490185e+19
This probably requires root to work properly and reads have been disabled for safety. Some MSRs are specified at the bit level, so reading in a int and converting to a float may be less than ideal. This collector is also different in that the value is passed in by the user.
Signed-off-by: Daniel Hodges [email protected]
Does it require root or not? But tbh I feel more and more perf stuff will need elevated permissions due to timing attack mitigration things so I'd be up for relaxing the non-root requirement but don't feel too strongly about it. @SuperQ wdyt?
Does it require root or not? But tbh I feel more and more perf stuff will need elevated permissions due to timing attack mitigration things so I'd be up for relaxing the non-root requirement but don't feel too strongly about it. @SuperQ wdyt?
Unfortunately it requires root, see the man page for /dev/cpu/<CPU>/msr
:
This file is protected so that it can be read and written only by the user root, or members of the group root.
This is probably far beyond the use case of most users, so I'm fine keeping this separate.
Unfortunately it requires root, see the man page for
/dev/cpu/<CPU>/msr
:This file is protected so that it can be read and written only by the user root, or members of the group root.
I'm probably stating the obvious here, but surely one could adjust the group ownership of those "files" and set them group-readable by e.g. a prometheus group.
I've done similar things in the past with sysfsutils to set various parts of /sys to be readable by specific non-root users at boot-time, e.g. the powercap / Intel RAPL stuff.
Yes, I've also used sysfsutils for similar permissions issues.
I'd be OK with this if we had some documentation on what sysfsutils config is needed for the collector(s) to work.
Needs rebase
The codespell issues are unrelated.
I'll be traveling for the next month or so and will have limited access, so if someone wants to rebase/fix codespell issues feel free.