[Feature]: Provide an example of exponential histogram.

[etep]

Describe the solution you'd like:

The examples include explicit bucket histogram but not exponential histogram. It's not clear if one needs to create a metric and then an aggregation, or if directly creating the aggregation is feasible. @jtescher

[cijothomas]

This is a very important capability, but not documented well. I was planning to defer this till 1.0 (strictly to keep scope in control!), so didn't spent much time on optimizing it either!

We can probably add support for picking Explicit vs Exponential using env variable, as I am unsure if we can keep the View mechanism to change aggregation. https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk_exporters/otlp.md#additional-environment-variable-configuration

If you have bandwidth to help, appreciate it. I'll keep the issue open to track this for stable release.

[etep]

@cijothomas -

Thank you for the note. Not sure how much bandwidth I have to pitch in, but there's a tentative interest on my side.

My experience & the docs.: First the story, why I'm here - I wanted to setup OTel metrics for a Rust project I work on. At the moment, I don't want to learn about the full scope of OTel, I just want (you know) an exponential histogram; data getting sent out, say, to a collector, then onward to a cloud service provider.

Unfortunately, the minimal setup for this is not well documented, and I haven't found (yet) any example projects to look at. Maybe there's something useful that I haven't found yet!

Thoughts on the impl. The project I'm working on claims to be very sensitive to perf. Regardless of the overall contribution to CPU time, we want all the code paths to be efficient. Exponential histogram has some good things for our use case, but, the impl. seems to expand the histogram by copying as the buckets start to fill in - see, e.g., this in the record method.

I'm curious to know if there is appetite for enabling a pre-allocated bucket structure or any other ideas here, e.g. append all samples to a vector and then allocate the buckets using min/max of that when the data is ready to be flushed. I'm sure there are other good ideas to be had around this topic - not sure if it will fit in with the project's needs, APIs, etc.

Thank you!

[cijothomas]

Unfortunately, the minimal setup for this is not well documented

You are right. The main reason is - Exponential Histogram is a very new thing in OTel world. It is not yet supported by all vendors, and the mapping to Prometheus is still experimental. In other words, using ExplicitBuckets is still the main scenario with Histograms today. Once ExponentialHistogram is more mainstream, it should get better treatment in docs/examples.

claims to be very sensitive to perf.

It is always a pleasure to hear such statements! We did a lot of work to optimize performance for Metrics aggregation recently, but Exponential Histograms were left out from that (to keep initial scope for 1.0 under control), but there are some active work going on now to make ExponentialHistograms also to get similar perf gains as other aggregations.

And yes, we welcome ideas to improve the performance of Histogram aggregations, including the idea of pre-allocating as much as possible! Would appreciate if you can open a separate issue with the suggestions, to better track them.

[otiv-emiel-vanseveren]

@cijothomas, could you provide a status update regarding this? Is this planned?

[scottgerring]

Some more details to work on an example of this:

• The open telemetry spec section on exponential histograms
• The advanced metrics sample where we can include this
• The opentelemetry-sdk implementation of the exponential histogram
• Cijo's link above explaining how to configure this

It looks like it is also possible to use metrics views and the feature flag spec_unstable_metrics_views to setup an SDK pipeline to transform explicit bucket histograms to exponential histograms.

@cijothomas you are very close to the metrics - is there anything else we'd want to do here (apart from the example)?

[cijothomas]

It looks like it is also possible to use metrics views and the feature flag spec_unstable_metrics_views to setup an SDK pipeline to transform explicit bucket histograms to exponential histograms.

^ This is the best route to take for the example now. The ENV variable one - Its best to not add support for this now, given OTel is moving towards new configuration model, and we don't want to add more ENV variables.

is there anything else we'd want to do here (apart from the example)?

For this issue - nothing more than an example. Outside of this, we want to stabilize spec_unstable_metrics_views too, which is a lot more involved, and is better tracked separately.

[kevindeng]

I too just wanted an exponential histogram and came across this. Following the comments above I was able to get it to work. Just gonna leave this here in case anyone else needs it.

    let meter_provider = SdkMeterProvider::builder()
        .with_resource(resource)
        .with_reader(reader)
        .with_view(|inst| {
            if let InstrumentKind::Histogram = inst.kind() {
                let s = Stream::builder()
                    .with_aggregation(Aggregation::Base2ExponentialHistogram {
                        max_size: 160,
                        max_scale: 20,
                        record_min_max: true,
                    })
                    .build()
                    .unwrap();
                Some(s)
            } else {
                None
            }
        })
        .build();

You need to enable the spec_unstable_metrics_views feature as mentioned:

opentelemetry_sdk = { version = "0.31.0", features = ["spec_unstable_metrics_views"] }