Add optional support for data type and description for metrics

[jlosito]

Prometheus metrics generally list a data type and a description of the metric. Some agents even look for this data. It generally looks like something below.

# HELP http_requests_total Total number of http api requests
# TYPE http_requests_total counter
http_requests_total{api="add_product"} 4633433

I believe this library doesn't add the data type nor the description which makes it difficult for some agents to scrape the data points.

Would it be possible to add the ability to denote a description and data type for each metric?

[cristaloleg]

which makes it difficult for some agents to scrape the data points.

Just curious, what agents fail to scrape?

[jlosito]

@cristaloleg here's a link to prometheus docs describing the format. The # TYPE lines actually matter and are consumed as tokens.

https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#comments-help-text-and-type-information

[jlosito]

I also found that the openmetrics standard also follows this format.

https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#metricfamily-metadata

[saez0pub]

@cristaloleg datadog agent drops the metrics if TYPE is not set, I have to map each metric to the correct type.

[tenmozes]

https://cloud.google.com/stackdriver/docs/managed-prometheus/troubleshooting - same applicable for GCP, so it's reasonable to add these annotations and option to control it

[hagen1778]

We should provide an option for user to decide whether TYPE needs to be added to the metrics exposition format. The option should make metrics lib compatible with DD agent and Google Managed Prometheus.

[cristaloleg]

@hagen1778 Is there any reason why TYPE should not be provided at all?

[hagen1778]

Is there any reason why TYPE should not be provided at all?

Yes, VictoriaMetrics ignores this information on ingestion. Since the main user of the metrics lib is VictoriaMetrics itself, it was decided to omit extra complexity.

[cristaloleg]

Ok, so just not implemeted instead of unachievable, thanks

[valyala]

Is there any reason why TYPE should not be provided at all?

The TYPE and HELP comments aren't provided by github.com/VictoriaMetrics/metrics package by default because of the following reasons:

• These comments are ignored by VictoriaMetrics and vmagent, as @hagen1778 said above.
• These comments require additional network bandwidth for transferring to metrics scraper such as Prometheus, VictoriaMetrics or vmagent. This may result in network bandwidth bottleneck in some setups.
• These comments may complicate usage of the github.com/VictoriaMetrics/metrics package, since the user need to pass proper HELP and TYPE values in the code somehow.
• These comments may contain invalid or misleading information. For example, the HELP line may contain irrelevant description for the metric, or the TYPE comment may contain invalid metric type (e.g. counter instead of gauge). The consumer of the /metrics page (either a human or metrics scraper) has no any protection against this, so it may break in various ways when consuming invalid data from these comments.

It is pity that some third-party scrapers require the TYPE or HELP comments given considerations mentioned above. The solution is to provide some kind of optional API at github.com/VictoriaMetrics/metrics, which could be used to instruct the package to generate some kind of TYPE and HELP comments per each exposed metric.

[mattmalec]

To anyone who's stuck on this like we were, we implemented support for HELP and TYPE so we could use the Datadog Agent to scrape Erigon metrics. It's built to be a drop-in replacement, so hopefully it shouldn't be hard for anyone else to use. Replacing the current implementation with ours was as simple as replacing the dependency in the go.mod file.

Hope this helps!

repo: https://github.com/ourzora/erigon-metrics see implementation: https://github.com/ourzora/erigon-metrics/commit/fa6fa517aec5cf6ab55ae785021fda4a0af22cab

[valyala]

Support for optional exposition of TYPE and HELP metadata has been added in the commit 9dc73588693ac062c917d59f038a17ad1bfcea65 . By default this metadata isn't exposed, since it isn't used by VictoriaMetrics components as explained in this comment. This commit adds an ability to enable exposing the TYPE and HELP metadata across all the metrics by calling metrics.ExposeMetadata(true):

• The TYPE metadata contains metric name plus metric type
• The HELP metadata contains metric name

This commit is included in the tag v1.28.0. Closing the feature request as done.

[AlekSi]

Created a follow-up issue: #109.

[makasim]

Historically, VictoriaMetrics didn't support metric metadata across all layers. Support for Prometheus style metric metadata was recently added in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2974. For the rationale behind adding metadata support, see:https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2974#issuecomment-2912971589. The metadata feature is disabled by default; set -enableMetadata=true flag to enable it.

Given that, it's time to add metadata support to the metrics library.

Key requirements:

• No breaking changes.
• Metadata must be opt-in and disabled by default. It should be accessible only through a separate API.
• Performance and potential data races must be evaluated and resolved before merging.

We should also consider adding TYPE and HELP metadata for VictoriaMetrics' own metrics. This metadata will not be exposed externally; it will be used internally for generating metrics documentation pages and other artifacts consumed by tooling or AI.