PV/Dataset metrics

[tahajahangir]

ZFS exposes several dataset metrics:

$ cat /proc/spl/kstat/zfs/ZPOOL NAME/objset-0x4c3
49 1 0x01 7 1904 882869614188 7661358045725488
name                            type data
dataset_name                    7    zfspv-pool/pvc-d1af69c6-b004-4df5-aa57-5e027884af59
writes                          4    162659962
nwritten                        4    169357302418427
reads                           4    19860562
nread                           4    20787773826774
nunlinks                        4    5326
nunlinked                       4    5326

Can we have this metrics per PV?

The related issue in prometheus/node_exporter for scraping these metrics: https://github.com/prometheus/node_exporter/issues/1602

[pawanpraka1]

@tahajahangir, if promethes's node exporter exposes it, then it should be available for ZFS-LocalPV also. Could you try the latest node exporter?

[tahajahangir]

I didn't understand the phrase " it should be available for ZFS-LocalPV". The node exporter metrics are scraped and available (like node_zfs_zpool_dataset_nread), but they contain only dataset-name (not namespace/pvc-name). We created a dashboard with expressions like this to join this data with pvc-name:

label_replace((rate(node_zfs_zpool_dataset_nread[1m])),"volumename", "$1", "dataset", ".*/(.*)" )
* on(volumename) group_left(persistentvolumeclaim, namespace)
kube_persistentvolumeclaim_info{cluster=~"$cluster", namespace=~"$namespace", persistentvolumeclaim=~"$volume"}

@pawanpraka1

[pawanpraka1]

This is great @tahajahangir. Adding this to our roadmap. Thanks.