[OpenTelemetry] Metrics derived from traces (Throughput, Latency and Errors) are not accurate when traces are sampled before being ingested by Elastic Observability

[cyrille-leclerc]

Enhancement Description

Metrics derived from traces (Throughput, Latency and Errors) are not accurate when traces are sampled before being ingested by Elastic Observability (ie by an OpenTelemetry Collector or OpenTelemetry APM agent or SDK).

@axw : "This is because there's no record of the sampling rate in span data; we would need open-telemetry/oteps#148 for this."

2021/11/18 OpenTelemetry specified how to propagate sampling data

• OTEP trace-168: Propagate parent sampling probability
• OTEP trace-170: Probability sampling of telemetry events

[tiny-dancer]

@cyrille-leclerc is there an update on this one? It looks like there's been merge activity related to the dependent otep in the last month or two. Thanks

[axw]

@tiny-dancer the proposal has been merged, but that's only the start of its journey. It still needs to be debated & accepted, converted to a specification, and then implemented. When all of those things have happened, we should be able to close this out.

[tiny-dancer]

Ah thank you for the clarification. One step down!

[MoshiBin]

Is there any workaround for when the sampling rate is 100%?

[axw]

@MoshiBin if the sampling rate is 100%, then the metrics should be accurate. This issue is only relevant when sampling < 100%.