filesystem /run mounts not excluded by default config when using path.rootfs

[johnl]

version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6

/bin/node_exporter --path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/host/root --web.listen-address=[0.0.0.0]:9100

I'm running node_exporter within kubernetes, using the community helm chart. It sets up the current mounts:

    volumeMounts:
    - mountPath: /host/proc
      name: proc
      readOnly: true
    - mountPath: /host/sys
      name: sys
      readOnly: true
    - mountPath: /host/root
      mountPropagation: HostToContainer
      name: root
      readOnly: true

  volumes:
  - hostPath:
      path: /proc
      type: ""
    name: proc
  - hostPath:
      path: /sys
      type: ""
    name: sys
  - hostPath:
      path: /
      type: ""
    name: root

metrics are being reported for filesystems mounted within the /run directory, despite being excluded by the default config, e.g:

hostname:/run/containerd/io.containerd.grpc.v1.cri/sandboxes/b7fc03b87b11f94e3ec41a04a9ef91f3159d37382369391b2e4159529c58808d/shm

ts=2022-08-08T15:09:42.577Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/)
ts=2022-08-08T15:09:42.577Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$

the mounts within the pod show up with the /host/root prefix, so I suspect the regular expression isn't matching.

shm on /host/root/run/containerd/io.containerd.grpc.v1.cri/sandboxes/b7fc03b87b11f94e3ec41a04a9ef91f3159d37382369391b2e4159529c58808d/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)

It would be nice if the path.rootfs prefix was taken into account by the filesystem excluding code, so users wouldn't have to

[discordianfish]

Only setting --path.rootfs=/host/root should be enough. This is used for sys and procfs as well. Can you see if that changes the behavior?

[johnl]

My apologies, I've actually just noticed that the default regular expression is for run/credentials/.+ which doesn't exclude these mountpoints, which are run/containerd...

For the record, if I tweak the the regular expression to be run.*, it works fine with the path.rootfs, and also works with path.procfs and path.sysfs in there at the same time. So this is entirely just me misreading the regular expression - sorry about that.

For what it's worth, I suspect that excluding everything in the run directory would probably be a good default, this regexp works for that:

    - --collector.filesystem.mount-points-exclude=^(dev|proc|run.*|sys|var/lib/docker/.+)($|/)

or, perhaps more contentiously, adding tmpfs to fs-types-exclude is another solution.

But obviously I can just fix this with a local config change.

I'm running node exporter within Kubernetes 1.22 on Ubuntu 20.04, installed with kubeadm, which should be a fairly common installation for people, so I expect this to show up more in future. Alternatively, I could suggest that the node exporter helm chart adds these configs locally, if you don't think they belong here.

[discordianfish]

Yeah I think the helmchart uses the other flags for some reason. Anyway, let's close.