[Bug]: missing attributes in prometheus exporter

[benkeil]

What happened?

Neither attributes defined by Resource::new() nor by versioned_meter() are exported correctly to prometheus.

let registry = Registry::new()

pub fn init_tracing() -> Result<(), ControllerError> {
    let tracer = opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(opentelemetry_otlp::new_exporter().tonic())
        .install_batch(opentelemetry_sdk::runtime::Tokio)
        .expect("Couldn't create OTLP tracer");

    let telemetry = tracing_opentelemetry::layer().with_tracer(tracer);

    tracing_subscriber::registry()
        .with(EnvFilter::from_default_env())
        .with(telemetry)
        .init();

    Ok(())
}

pub fn init_meter(registry: &Registry) -> Result<Meter, ControllerError> {
    let exporter = opentelemetry_prometheus::exporter()
        .with_registry(registry.clone())
        .build()
        .map_err(|_| ControllerError::ConfigurationError)?;

    let provider = MeterProvider::builder()
        .with_reader(exporter)
        .with_resource(Resource::new(vec![
            KeyValue::new(
                "environment".to_string(),
                std::env::var("APP_ENVIRONMENT").unwrap_or("local".to_string()),
            ),
            KeyValue::new("app".to_string(), "github-operator".to_string()),
        ]))
        .build();

    Ok(provider.versioned_meter(
        "github-operator",
        None::<&'static str>,
        None::<&'static str>,
        Some(vec![
            KeyValue::new(
                "environment".to_string(),
                std::env::var("APP_ENVIRONMENT").unwrap_or("local".to_string()),
            ),
            KeyValue::new("app".to_string(), "github-operator".to_string()),
        ]),
    ))
}

use prometheus::{Encoder, Registry, TextEncoder};

async fn metrics_handler(State(HttpState { registry }): State<HttpState>) -> impl IntoResponse {
    let encoder = TextEncoder::new();
    let metric_families = registry.gather();
    let mut result = Vec::new();
    encoder
        .encode(&metric_families, &mut result)
        .map_err(ControllerError::PrometheusError)
        .expect("Couldn't encode metrics");

    ([(header::CONTENT_TYPE, "text/plain")], result)
}

produces:

# HELP operator_results_total reconciliation results of the operator
# TYPE operator_results_total counter
operator_results_total{controller="repository-github-controller",status="ok",otel_scope_name="github-operator"} 3
# HELP otel_scope_info Instrumentation Scope metadata
# TYPE otel_scope_info gauge
otel_scope_info{otel_scope_name="github-operator"} 1
# HELP target_info Target metadata
# TYPE target_info gauge
target_info{app="github-operator",environment="local"} 1

API Version

0.21

SDK Version

0.21.2

What Exporters are you seeing the problem on?

No response

Relevant log output

No response

[protochron]

I'm seeing the same thing relaying metrics to an OTEL collector. Both resources and the attributes on versioned_meter() are not being serialized and sent in 0.21.2. I haven't tried an older version or been able to upgrade yet because the tracing crate is still on 0.21.

[cijothomas]

I'm seeing the same thing relaying metrics to an OTEL collector. Both resources and the attributes on versioned_meter() are not being serialized and sent in 0.21.2. I haven't tried an older version or been able to upgrade yet because the tracing crate is still on 0.21.

Could you share a minimal repro? I am not able to repro it. let meter = global::meter_with_version("ex.com/basic", Some("0.1.0"), Some("schema"), Some(vec![KeyValue::new("key1", "value1")])); produces the following in collector:

[protochron]

@cijothomas It looks like was missing an option in my otel collector config so it seemed like they were being dropped. Enabling resource_to_telemetry_conversion added the resource labels as expected, but anything in InstrumentationScope is only available on the collector and would need to be added in with a transform afaik.

[TommyCpp]

The resource is currently emitted as target_info metrics instead of attached as metrics attributes.

The spec

A Prometheus Exporter MAY offer configuration to add resource attributes as metric attributes. By default, it MUST NOT add any resource attributes as metric attributes.