Filtering metrics by tags

[yorugac]

To avoid quickly overshooting server side limits, it would be good to have configurable option on how many metrics and how many labels are sent via configuration and perhaps, which ones are sent in the first place.

Partially related k6 issue

[codebien]

@yorugac is this feature required on the output side? I think the output could just rely on the SystemTags provided by k6. If the user wants to reduce the number of tags delivered s/he can just disable some of them with the option. For the same reason, I think we should remove these options from the configuration.

WDYT?

[yorugac]

I'd say this feature is needed for outputs in general and probably should be part of the core :sweat_smile:

This issue is not about system tags but about more flexible configuration in general, including custom tags. E.g. Prometheus allows quite a versatile configuration for which metrics and labels to keep and which to discard. But that's configuration for scraping, remote write endpoint doesn't have that, AFAIK. Plus, the issue of slow metrics processing from k6 side has been imposing limits on how much can be processed so filtering seemed like a good addition. That's how this issue appeared initially.

Now that I look at it, it can be read as a duplicate for https://github.com/grafana/k6/issues/1321. If so, then it makes sense to close this one: this feature request is more fitting to k6 core anyway. Just to be clear, I consider this a valid feature request regardless of metrics refactoring: as a user, I don't always want to have all metrics stored and system tags option is insufficient.

I think we should remove these options from the configuration.

For sure. These were really just temporary options until something better appears.

[yorugac]

remote write endpoint doesn't have that, AFAIK

Correction: remote write endpoint can probably rely on metric_relabel_configs option to filter metrics received from k6. But that'd be on Prometheus' side, so after metrics were already sent by k6 but before they are stored. And this option may not be present for other remote write outputs. Still, better than nothing.

[smazurov]

why is systemTags insufficient for basic pairing down of whats sent?

[codebien]

Currently, we're not planning to implement this. As we mentioned, we want to address it with a wider approach. If required, use one of the collector available out there to filter out metrics or use one of the previously mentioned workarounds.