Problem with textfile collector combined with --web.config.file option

[nekatskin]

Host operating system: output of uname -a

Linux 4.15.0-197-generic x86_64

node_exporter version: output of node_exporter --version

node_exporter, version 1.7.0 (branch: HEAD, revision: 7333465abf9efba81876303bb57e6fadb946041b)
  build user:       root@35918982f6d8
  build date:       20231112-23:53:35
  go version:       go1.21.4
  platform:         linux/amd64
  tags:             netgo osusergo static_build

node_exporter command line flags

/usr/local/bin/node_exporter \
    '--collector.textfile.directory=/var/lib/node_exporter/textfile_collector' \
    '--collector.perf' \
    '--collector.processes' \
    '--collector.interrupts' \
    '--web.config.file=/etc/node_exporter/config.yaml' \
    '--web.listen-address=0.0.0.0:9100' \
    '--web.telemetry-path=/metrics'

node_exporter log output

Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com systemd[1]: Stopping Prometheus Node Exporter...
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com systemd[1]: Stopped Prometheus Node Exporter.
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com systemd[1]: Started Prometheus Node Exporter.
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.300Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.302Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.304Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.304Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed fl
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.305Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed fl
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.307Z caller=node_exporter.go:110 level=info msg="Enabled collectors"
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.307Z caller=node_exporter.go:117 level=info collector=arp
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.307Z caller=node_exporter.go:117 level=info collector=bcache
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.307Z caller=node_exporter.go:117 level=info collector=bonding
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.308Z caller=node_exporter.go:117 level=info collector=btrfs
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.308Z caller=node_exporter.go:117 level=info collector=conntrack
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.308Z caller=node_exporter.go:117 level=info collector=cpu
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.308Z caller=node_exporter.go:117 level=info collector=cpufreq
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.308Z caller=node_exporter.go:117 level=info collector=diskstats
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.309Z caller=node_exporter.go:117 level=info collector=dmi
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.309Z caller=node_exporter.go:117 level=info collector=edac
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.309Z caller=node_exporter.go:117 level=info collector=entropy
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.309Z caller=node_exporter.go:117 level=info collector=fibrechannel
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.309Z caller=node_exporter.go:117 level=info collector=filefd
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.309Z caller=node_exporter.go:117 level=info collector=filesystem
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.310Z caller=node_exporter.go:117 level=info collector=hwmon
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.310Z caller=node_exporter.go:117 level=info collector=infiniband
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.310Z caller=node_exporter.go:117 level=info collector=interrupts
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.310Z caller=node_exporter.go:117 level=info collector=ipvs
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.311Z caller=node_exporter.go:117 level=info collector=loadavg
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.311Z caller=node_exporter.go:117 level=info collector=mdadm
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.311Z caller=node_exporter.go:117 level=info collector=meminfo
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.312Z caller=node_exporter.go:117 level=info collector=netclass
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.312Z caller=node_exporter.go:117 level=info collector=netdev
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.312Z caller=node_exporter.go:117 level=info collector=netstat
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.312Z caller=node_exporter.go:117 level=info collector=nfs
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.312Z caller=node_exporter.go:117 level=info collector=nfsd
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.312Z caller=node_exporter.go:117 level=info collector=nvme
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.313Z caller=node_exporter.go:117 level=info collector=os
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.313Z caller=node_exporter.go:117 level=info collector=perf
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.313Z caller=node_exporter.go:117 level=info collector=powersupplyclass
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.313Z caller=node_exporter.go:117 level=info collector=pressure
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.313Z caller=node_exporter.go:117 level=info collector=processes
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.313Z caller=node_exporter.go:117 level=info collector=rapl
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=schedstat
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=selinux
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=sockstat
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=softnet
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=stat
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=tapestats
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=textfile
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=thermal_zone
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=time
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=timex
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=udp_queues
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=uname
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=vmstat
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=xfs
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.314Z caller=node_exporter.go:117 level=info collector=zfs
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.316Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100
Sep 27 11:59:47 eu059.iplb.airstore.scal3fl3x.com node_exporter[4156]: ts=2024-09-27T11:59:47.317Z caller=tls_config.go:313 level=info msg="TLS is disabled." http2=false address

Are you running node_exporter in Docker?

No

What did you do that produced an error?

I tried to launch node_exporter with the textfile collector because I need some custom metrics. When I start the node_exporter with textfile collector directory option, it's working fine I'm able to to retrieve my custom metrics. But whenever I tried to start the node_exporter with both the --web-config-file option and --collector.textfile.directory option, I can't see anymore my custom metrics

What did you expect to see?

I expect to be able to have my custom metrics while using the --web-config-file option + --collector.textfile.directory option

What did you see instead?

The custom metrics are not scrapped anymore while using --web-config-file option + --collector.textfile.directory option

[dswarbrick]

This is quite surprising, since those flags are completely unrelated, and defined in totally different parts of the code. The --web.config.file flag is technically not even part of node_exporter - it is provided by the prometheus/exporter-toolkit package.

Have you tried to reproduce this with the latest version of node_exporter, v1.8.2?

PS: what the heck is still running kernel 4.15? It was EOL'd in April 2018, more than six years ago.

[SuperQ]

PS: what the heck is still running kernel 4.15?

Ubuntu 18.04, based on the uname.