node_exporter
node_exporter copied to clipboard
Published
20 hours ago •
prometheus
prometheus
fatal error: unexpected signal during runtime execution
Host operating system: output of lsb_release -a and uname -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
Linux xb 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
node_exporter version: output of node_exporter --version
node_exporter, version 1.3.1 (branch: debian/sid, revision: 1.3.1-1)
build user: [email protected]
build date: 20220114-23:26:34
go version: go1.17.3
platform: linux/amd64
node_exporter command line flags
/usr/bin/prometheus-node-exporter
Are you running node_exporter in Docker?
No
Description of the problem?
I noticed that the service had restarted with the following log output. Maybe some of the developers can make anything of it.
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: fatal error: unexpected signal during runtime execution
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: [signal SIGSEGV: segmentation violation code=0x2 addr=0xc000122000 pc=0xc000122000]
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime stack:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime: unexpected return pc for runtime.sigpanic called from 0xc000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: stack: frame={sp:0x7f8550f78c40, fp:0x7f8550f78c90} stack=[0x7f85507791d0,0x7f8550f78dd0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b40: 0x01007f8550f78b60 0x0000000000000004
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b50: 0x000000000000001f 0x000000c000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b60: 0x000000c000122000 0x0000000000000002
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b70: 0x0000000000437a91 <runtime.throw+0x0000000000000071> 0x00007f8550f78c10
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b80: 0x0000000000a84463 0x00007f8550f78bc8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b90: 0x0000000000437d48 <runtime.fatalthrow.func1+0x0000000000000048> 0x000000c0001ff520
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78ba0: 0x0000000000000001 0x0000000000000001
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bb0: 0x00007f8550f78c10 0x0000000000437a91 <runtime.throw+0x0000000000000071>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bc0: 0x000000c0001ff520 0x00007f8550f78c00
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bd0: 0x0000000000437cd0 <runtime.fatalthrow+0x0000000000000050> 0x00007f8550f78be0
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78be0: 0x0000000000437d00 <runtime.fatalthrow.func1+0x0000000000000000> 0x000000c0001ff520
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bf0: 0x0000000000437a91 <runtime.throw+0x0000000000000071> 0x00007f8550f78c10
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c00: 0x00007f8550f78c30 0x0000000000437a91 <runtime.throw+0x0000000000000071>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c10: 0x00007f8550f78c18 0x0000000000437ac0 <runtime.throw.func1+0x0000000000000000>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c20: 0x0000000000a8fbd5 0x000000000000002a
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c30: 0x00007f8550f78c80 0x000000000044dbd6 <runtime.sigpanic+0x0000000000000396>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c40: <0x0000000000a8fbd5 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c50: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c60: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c70: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c80: 0x00007f8550f78d10 !0x000000c000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c90: >0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78ca0: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cb0: 0x0000000000000000 0x0020300000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cc0: 0x0000000350f78ce0 0x0000000000426f91 <runtime.putempty+0x0000000000000031>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cd0: 0x0000000000000000 0x0000000000002000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78ce0: 0x0000000000001658 0x000000c000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cf0: 0x000000c00050f800 0x00007f856ae50fff
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d00: 0x00007f856ac5a1b2 0x000000c00050f800
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d10: 0x00007f8550f78d68 0x00000000004205da <runtime.gcDrain+0x00000000000001ba>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d20: 0x000000c000034698 0x000000c000034698
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d30: 0x000000c0001ff520 0x0000000000007bf8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d40: 0x0000000000000000 0xfffffffffffffde8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d50: 0x0000000000000000 0x000000c000383ba0
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d60: 0x0000000000aaccc8 0x00007f8550f78db8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d70: 0x000000000041d7ae <runtime.gcBgMarkWorker.func2+0x000000000000006e> 0x000000c000034698
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d80: 0x0000000000000007 0x0000000000000100
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime.throw({0xa8fbd5, 0x0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/panic.go:1198 +0x71
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime: unexpected return pc for runtime.sigpanic called from 0xc000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: stack: frame={sp:0x7f8550f78c40, fp:0x7f8550f78c90} stack=[0x7f85507791d0,0x7f8550f78dd0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b40: 0x01007f8550f78b60 0x0000000000000004
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b50: 0x000000000000001f 0x000000c000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b60: 0x000000c000122000 0x0000000000000002
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b70: 0x0000000000437a91 <runtime.throw+0x0000000000000071> 0x00007f8550f78c10
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b80: 0x0000000000a84463 0x00007f8550f78bc8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78b90: 0x0000000000437d48 <runtime.fatalthrow.func1+0x0000000000000048> 0x000000c0001ff520
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78ba0: 0x0000000000000001 0x0000000000000001
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bb0: 0x00007f8550f78c10 0x0000000000437a91 <runtime.throw+0x0000000000000071>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bc0: 0x000000c0001ff520 0x00007f8550f78c00
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bd0: 0x0000000000437cd0 <runtime.fatalthrow+0x0000000000000050> 0x00007f8550f78be0
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78be0: 0x0000000000437d00 <runtime.fatalthrow.func1+0x0000000000000000> 0x000000c0001ff520
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78bf0: 0x0000000000437a91 <runtime.throw+0x0000000000000071> 0x00007f8550f78c10
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c00: 0x00007f8550f78c30 0x0000000000437a91 <runtime.throw+0x0000000000000071>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c10: 0x00007f8550f78c18 0x0000000000437ac0 <runtime.throw.func1+0x0000000000000000>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c20: 0x0000000000a8fbd5 0x000000000000002a
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c30: 0x00007f8550f78c80 0x000000000044dbd6 <runtime.sigpanic+0x0000000000000396>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c40: <0x0000000000a8fbd5 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c50: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c60: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c70: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c80: 0x00007f8550f78d10 !0x000000c000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78c90: >0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78ca0: 0x0000000000000000 0x0000000000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cb0: 0x0000000000000000 0x0020300000000000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cc0: 0x0000000350f78ce0 0x0000000000426f91 <runtime.putempty+0x0000000000000031>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cd0: 0x0000000000000000 0x0000000000002000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78ce0: 0x0000000000001658 0x000000c000122000
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78cf0: 0x000000c00050f800 0x00007f856ae50fff
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d00: 0x00007f856ac5a1b2 0x000000c00050f800
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d10: 0x00007f8550f78d68 0x00000000004205da <runtime.gcDrain+0x00000000000001ba>
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d20: 0x000000c000034698 0x000000c000034698
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d30: 0x000000c0001ff520 0x0000000000007bf8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d40: 0x0000000000000000 0xfffffffffffffde8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d50: 0x0000000000000000 0x000000c000383ba0
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d60: 0x0000000000aaccc8 0x00007f8550f78db8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d70: 0x000000000041d7ae <runtime.gcBgMarkWorker.func2+0x000000000000006e> 0x000000c000034698
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: 0x00007f8550f78d80: 0x0000000000000007 0x0000000000000100
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime.sigpanic()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/signal_unix.go:719 +0x396
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 65 [GC worker (idle)]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime.systemstack_switch()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/asm_amd64.s:350 fp=0xc000402f60 sp=0xc000402f58 pc=0x467ee0
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime.gcBgMarkWorker()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/mgc.go:1256 +0x1b1 fp=0xc000402fe0 sp=0xc000402f60 pc=0x41d491
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: runtime.goexit()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/asm_amd64.s:1581 +0x1 fp=0xc000402fe8 sp=0xc000402fe0 pc=0x46a101
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by runtime.gcBgMarkStartWorkers
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/mgc.go:1124 +0x25
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 1 [IO wait, 8514 minutes]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.runtime_pollWait(0x7f856915db30, 0x72)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/netpoll.go:234 +0x89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).wait(0xc000301f80, 0x66, 0x0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:84 +0x32
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).waitRead(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*FD).Accept(0xc000301f80)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_unix.go:402 +0x22c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*netFD).accept(0xc000301f80)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/fd_unix.go:173 +0x35
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*TCPListener).accept(0xc00030db90)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/tcpsock_posix.go:140 +0x28
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*TCPListener).Accept(0xc00030db90)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/tcpsock.go:262 +0x3d
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.(*Server).Serve(0xc0003b6000, {0xb4dfa0, 0xc00030db90})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:3001 +0x394
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/exporter-toolkit/web.Serve({0xb4dfa0, 0xc00030db90}, 0xc0003b6000, {0x0, 0xe0}, {0xb42c80, 0xc0000ee4c0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/exporter-toolkit/web/tls_config.go:196 +0x17f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/exporter-toolkit/web.ListenAndServe(0xc0003b6000, {0x0, 0x0}, {0xb42c80, 0xc0000ee4c0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/exporter-toolkit/web/tls_config.go:188 +0xf9
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: main.main()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/node_exporter.go:201 +0x137b
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 56 [select]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/client_golang/prometheus.(*Registry).Gather(0xc000321ae0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/registry.go:513 +0x945
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/client_golang/prometheus.Gatherers.Gather({0xc000305960, 0x2, 0x2})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/registry.go:719 +0x217
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/client_golang/prometheus/promhttp.HandlerFor.func1({0x7f85680d63f8, 0xc0001ac640}, 0xc0002d4200)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/promhttp/http.go:126 +0x108
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.HandlerFunc.ServeHTTP(0xb446a0, {0x7f85680d63f8, 0xc0001ac640}, 0xc0004d23f0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:2046 +0x2f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1({0x7f85680d63f8, 0xc0001ac640}, 0xc0001ac640)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:40 +0xd4
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.HandlerFunc.ServeHTTP(0xb4e1b0, {0x7f85680d63f8, 0xc0001ac640}, 0x40)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:2046 +0x2f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1({0xb4e1b0, 0xc0003b61c0}, 0xc0002d4200)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:101 +0x92
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.HandlerFunc.ServeHTTP(0xc0001ac5f0, {0xb4e1b0, 0xc0003b61c0}, 0x2)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:2046 +0x2f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: main.(*handler).ServeHTTP(0xc0000ee5c0, {0xb4e1b0, 0xc0003b61c0}, 0xc0002d4200)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/node_exporter.go:80 +0x3f2
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.(*ServeMux).ServeHTTP(0x0, {0xb4e1b0, 0xc0003b61c0}, 0xc0002d4200)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:2424 +0x149
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.serverHandler.ServeHTTP({0xc000307b90}, {0xb4e1b0, 0xc0003b61c0}, 0xc0002d4200)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:2878 +0x43b
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.(*conn).serve(0xc000312b40, {0xb52dd8, 0xc000307aa0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:1929 +0xb08
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by net/http.(*Server).Serve
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:3033 +0x4e8
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765295 [IO wait]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.runtime_pollWait(0x7f856918a480, 0x72)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/netpoll.go:234 +0x89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).wait(0xc000432900, 0xc0002e11d0, 0x0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:84 +0x32
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).waitRead(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*FD).ReadMsg(0xc000432900, {0xc0002e11d0, 0x10, 0x10}, {0xc00028a020, 0x1000, 0x1000}, 0x1)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_unix.go:250 +0x31c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*netFD).readMsg(0xc000432900, {0xc0002e11d0, 0xb5ae10, 0x1}, {0xc00028a020, 0x0, 0x26d6a}, 0xc00042a0a0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/fd_posix.go:68 +0x37
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*UnixConn).readMsg(0xc00031eea8, {0xc0002e11d0, 0xc000280000, 0xb42860}, {0xc00028a020, 0xb42860, 0xc0004f4990})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/unixsock_posix.go:116 +0x4f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*UnixConn).ReadMsgUnix(0xc00031eea8, {0xc0002e11d0, 0x40f3b4, 0x467f2e}, {0xc00028a020, 0xc000685da0, 0x43a620})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/unixsock.go:143 +0x3c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.(*oobReader).Read(0xc00028a000, {0xc0002e11d0, 0xc000685e20, 0x40f707})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/transport_unix.go:21 +0x45
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: io.ReadAtLeast({0xb42da0, 0xc00028a000}, {0xc0002e11d0, 0x10, 0x10}, 0x10)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011io/io.go:328 +0x9a
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: io.ReadFull(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011io/io.go:347
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.(*unixTransport).ReadMessage(0xc00048c2e8)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/transport_unix.go:91 +0x11e
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.(*Conn).inWorker(0xc0003b62a0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/conn.go:348 +0x36
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/godbus/dbus.(*Conn).Auth
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/auth.go:118 +0x9e5
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765337 [runnable]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: syscall.Syscall6(0x106, 0xffffffffffffff9c, 0xc0002ea120, 0xc00049a6b8, 0x100, 0x0, 0x0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011syscall/asm_linux_amd64.s:43 +0x5
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: syscall.fstatat(0xc000279578, {0xc0002ea0a0, 0x45270f}, 0xc00049a6b8, 0xc000279548)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011syscall/zsyscall_linux_amd64.go:1441 +0x10f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: syscall.Lstat(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011syscall/syscall_linux_amd64.go:74
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: os.lstatNolog.func1(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011os/stat_unix.go:46
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: os.ignoringEINTR(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011os/file_posix.go:246
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: os.lstatNolog({0xc0002ea0a0, 0x5})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011os/stat_unix.go:45 +0x5b
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: os.Lstat({0xc0002ea0a0, 0x19})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011os/stat.go:22 +0x34
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: os.(*File).readdir(0xc000010bd0, 0xffffffffffffffff, 0x2)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011os/dir_unix.go:129 +0x70b
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: os.(*File).Readdir(0xc00009eb40, 0x13)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011os/dir.go:41 +0x25
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: io/ioutil.ReadDir({0xc00009eb40, 0x101000000000008})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011io/ioutil/ioutil.go:63 +0x45
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/procfs/sysfs.parseNetClassIface({0xc00009eb40, 0x13})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/procfs/sysfs/net_class.go:124 +0x65
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/procfs/sysfs.FS.NetClassByIface({{0xa6aa9a, 0x0}}, {0xc00031a117, 0x0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/procfs/sysfs/net_class.go:89 +0x112
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.(*netClassCollector).getNetClassInfo(0xc0000ee7c0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/netclass_linux.go:196 +0x12f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.(*netClassCollector).Update(0xc0000ee7c0, 0xc000500000)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/netclass_linux.go:66 +0x3e
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.execute({0xa6f33c, 0xc000000001}, {0xb432e0, 0xc0000ee7c0}, 0xc00040bfd0, {0xb42c80, 0xc0000ee4c0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/collector.go:161 +0x9c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.NodeCollector.Collect.func1({0xa6f33c, 0xc00069f8f0}, {0xb432e0, 0xc0000ee7c0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/collector.go:152 +0x3d
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/prometheus/node_exporter/collector.NodeCollector.Collect
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/collector.go:151 +0xd5
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765315 [semacquire]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: sync.runtime_Semacquire(0xc00002cbe0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/sema.go:56 +0x25
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: sync.(*WaitGroup).Wait(0xc0006e02a0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011sync/waitgroup.go:130 +0x71
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/registry.go:463 +0x2f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/registry.go:462 +0x56f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765356 [runnable]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: regexp.(*inputString).step(0xc0000ec4a0, 0xc)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011regexp/regexp.go:383 +0x8d
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: regexp.(*Regexp).tryBacktrack(0xc00021ce60, 0xc0000ec420, {0xb56e98, 0xc0000ec4a0}, 0xf52f80, 0x0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011regexp/backtrack.go:218 +0xb42
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: regexp.(*Regexp).backtrack(0xc00021ce60, {0x0, 0x0, 0x0}, {0xc0002ee720, 0x12}, 0x0, 0xc0002ee720, {0xf52f80, 0x0, ...})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011regexp/backtrack.go:326 +0x1e9
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: regexp.(*Regexp).doExecute(0x50, {0x0, 0x0}, {0x0, 0x80, 0x80}, {0xc0002ee720, 0x100000000000000}, 0x7f856891b0e0, 0x0, ...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011regexp/exec.go:535 +0x3a9
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: regexp.(*Regexp).doMatch(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011regexp/exec.go:514
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: regexp.(*Regexp).MatchString(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011regexp/regexp.go:525
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.filterUnits({0xc000646000, 0x283, 0x4}, 0x2, 0xc00023cd78, {0xb42be0, 0xc0000af360})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/systemd_linux.go:461 +0x18c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.(*systemdCollector).Update(0xc000332000, 0xc000500120)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/systemd_linux.go:184 +0x5f4
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.execute({0xa6e782, 0xc00000000e}, {0xb43580, 0xc000332000}, 0xc00023cfd0, {0xb42c80, 0xc0000ee4c0})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/collector.go:161 +0x9c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.NodeCollector.Collect.func1({0xa6e782, 0xc000692e10}, {0xb43580, 0xc000332000})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/collector.go:152 +0x3d
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/prometheus/node_exporter/collector.NodeCollector.Collect
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/collector.go:151 +0xd5
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765316 [semacquire]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: sync.runtime_Semacquire(0x0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/sema.go:56 +0x25
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: sync.(*WaitGroup).Wait(0xc00040e690)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011sync/waitgroup.go:130 +0x71
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/node_exporter/collector.NodeCollector.Collect({0xc00022c330, {0xb42c80, 0xc0000ee4c0}}, 0xc000500120)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/node_exporter/collector/collector.go:156 +0x254
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/registry.go:446 +0x102
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/prometheus/client_golang/prometheus/registry.go:538 +0xb4d
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765303 [IO wait]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.runtime_pollWait(0x7f856915dc18, 0x72)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/netpoll.go:234 +0x89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).wait(0xc0003c8000, 0xc000307ba1, 0x0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:84 +0x32
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).waitRead(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*FD).Read(0xc0003c8000, {0xc000307ba1, 0x1, 0x1})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_unix.go:167 +0x25a
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*netFD).Read(0xc0003c8000, {0xc000307ba1, 0x9d5a40, 0xc000404f48})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/fd_posix.go:56 +0x29
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*conn).Read(0xc00031e120, {0xc000307ba1, 0xc000317390, 0xc0001ac550})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/net.go:183 +0x45
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net/http.(*connReader).backgroundRead(0xc000307b90)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:672 +0x3f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by net/http.(*connReader).startBackgroundRead
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/http/server.go:668 +0xcf
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765297 [chan receive]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.newConn.func1()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/conn.go:253 +0x32
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/godbus/dbus.newConn
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/conn.go:252 +0x17b
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765378 [IO wait]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.runtime_pollWait(0x7f856918a2b0, 0x72)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011runtime/netpoll.go:234 +0x89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).wait(0xc00007e500, 0xc000309da0, 0x0)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:84 +0x32
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*pollDesc).waitRead(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_poll_runtime.go:89
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: internal/poll.(*FD).ReadMsg(0xc00007e500, {0xc000309da0, 0x10, 0x10}, {0xc00028c620, 0x1000, 0x1000}, 0xc000495320)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011internal/poll/fd_unix.go:250 +0x31c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*netFD).readMsg(0xc00007e500, {0xc000309da0, 0xb5ae10, 0x1}, {0xc00028c620, 0x0, 0x0}, 0xc0005bfe48)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/fd_posix.go:68 +0x37
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*UnixConn).readMsg(0xc000010270, {0xc000309da0, 0xc0005bfec0, 0xb42860}, {0xc00028c620, 0xb42860, 0xc000064690})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/unixsock_posix.go:116 +0x4f
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: net.(*UnixConn).ReadMsgUnix(0xc000010270, {0xc000309da0, 0x40f3b4, 0x467f2e}, {0xc00028c620, 0xc0004bbda0, 0x43a620})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011net/unixsock.go:143 +0x3c
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.(*oobReader).Read(0xc00028c600, {0xc000309da0, 0xc0004bbe20, 0x40f707})
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/transport_unix.go:21 +0x45
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: io.ReadAtLeast({0xb42da0, 0xc00028c600}, {0xc000309da0, 0x10, 0x10}, 0x10)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011io/io.go:328 +0x9a
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: io.ReadFull(...)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011io/io.go:347
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.(*unixTransport).ReadMessage(0xc0005bebb8)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/transport_unix.go:91 +0x11e
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.(*Conn).inWorker(0xc0003b6380)
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/conn.go:348 +0x36
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/godbus/dbus.(*Conn).Auth
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/auth.go:118 +0x9e5
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765362 [chan receive]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/godbus/dbus.newConn.func1()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/conn.go:253 +0x32
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/godbus/dbus.newConn
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/godbus/dbus/conn.go:252 +0x17b
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: goroutine 9765381 [chan receive]:
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: github.com/coreos/go-systemd/dbus.(*Conn).dispatch.func1()
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/coreos/go-systemd/dbus/subscription.go:55 +0x52
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: created by github.com/coreos/go-systemd/dbus.(*Conn).dispatch
Sep 10 01:56:28 xb prometheus-node-exporter[1415]: #011github.com/coreos/go-systemd/dbus/subscription.go:53 +0xc5
Sep 10 01:56:29 xb systemd[1]: Stopped Prometheus exporter for machine metrics.
Sep 10 01:56:29 xb systemd[1]: Started Prometheus exporter for machine metrics.
What did you do that produced an error?
Nothing - service crashed and restarted. No problems seen after systemd restarted the service.
Further notes
I switched to server ECC memory, after I had experienced some kernel crashes (see this issue).
