High Cardinality Metrics - Prometheus Remote Write (experimental-prometheus-rw)

[jameshounshell]

Feature Description

When using the Prometheus Remote Write functionality it is not currently possible to limit what labels are included in the series sent to prometheus. Currently we have an issue where the full url is included in the k6_http_ prefixed metrics where the unique id's randomly generated by some of our developers tests absolutely explode the cardinality leading to poor performance when querying or when the developers use the k6 grafana dashboard for time spans longer than a few minutes.

For example this urls like these result in high cardinality within prometheus http://redacted.redacted.svc.cluster.local:8080/api/v1/applications/2b38e5b6-7e72-41a2-9f4d-9f2e51787e78

I could ask the developers to use a smaller fixed number of ID's but this doesn't seem feasible in the long run.

Suggested Solution (optional)

I'm looking for something similar to Prometheus' relabel config functionality (regex matching and manipulation) or have the k6 library have some way to mark/sanitize urls containing unique ID's (ex: opentelemetry tracing does this automatically based on common http server frameworks).

Alternatively even some command line flags to drop certain labels from the emitted metrics would be helpful.

Already existing or connected issues / PRs (optional)

No response

Edit/Update

• I found this in the k6 documentation about how to limit the name label but I'd need to test if this can also be used to edit/limit the url label. Though this still leaves it to the discretion of the the developer and their test, rather than allowing me to override the default behavior.