grafana
Expose disk usage metrics
Is your feature request related to a problem? Please describe.
We just set up loki. How do we know if we have enough disk space allocated for the various components? How do we monitor (and possibly alert) when we should increase disk sizes?
Describe the solution you'd like
Loki exposes metrics on total, available and used bytes and inodes on it's data volumes.
Describe alternatives you've considered
Get disk utilization metrics for Persistent Volumes directly from K8s, but that has not been possible for a few years now: https://github.com/google/cadvisor/issues/1702
View disk usage in Azure, but that is not possible either.
Run kubectl exec -n loki grafana-loki-loki-distributed-ingester-0 df manually every day and plot the results in a spreadsheet? But oh lord plz let us avoid this.
On my Kubernetes I have metrics like kubelet_volume_stats_used_bytes, kubelet_volume_stats_inodes, etc.
