Ivan Rizzante

Hi @anuraaga I can't provide the actual prometheus HTTP output but the out-of-order exemplars seems to be disappeared on their own from the logs. It looks like there may have...

Hi @anuraaga I managed to reproduce the problem and here is the actual http output [httpoutput.txt](https://github.com/open-telemetry/opentelemetry-java/files/8141624/httpoutput.txt) You can see lines like these: ``` http_client_duration_bucket{http_flavor="1.1",http_method="POST",http_status_code="200",net_peer_name="xxx",net_peer_port="xxxx",le="1000.0"} 2.0 1645796601.542 # {span_id="8575abe2de3a34a3",trace_id="0d41134e674c5bfe5892a1c9e36e4019"} 804.039339 1645794322.137...

Any update on the above?

@jkwatson if you're asking me no I'm not able to do that, IMHO a possible fix could be removing timestamps from metrics.

I confirm the issue is still present with version 1.12.0, it would be nice if someone could help fixing this issue or eventually address me to where I should make...

@jkwatson actually Prometheus acts just as a backend for OpenTelemetry metrics format, the way metrics with exemplars should look like is defined in OpenTelemetry [specs](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#exemplars). Prometheus is complaining about out...

@jkwatson in the [specs](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#exemplars) I reported you can read: > Exemplars MUST consist of a LabelSet and a value, and MAY have a timestamp So having a timestamp is not...

OpenTelemetry spec is not for Prometheus only so I don't understand why it should be reported there. Regardless of this, the removal of timestamps from exemplars doesn't depend on mentioning...

Hi @anuraaga sorry but the out-of-order exemplars are produced by Prometheus exporter not by Prometheus, which only complains about what it finds wrong. If you check this [comment](https://github.com/open-telemetry/opentelemetry-java/issues/4193#issuecomment-1050869476) you can...

@anuraaga in the same comment I mentioned you can find the raw output of the whole scrape. You can see yourself that given one histogram ``` http_client_duration_count{http_flavor="1.1",http_method="GET",http_status_code="200",net_peer_name="xxx",net_peer_port="xxxx"} 3.0 1645796601.541 http_client_duration_sum{http_flavor="1.1",http_method="GET",http_status_code="200",net_peer_name="xxx",net_peer_port="xxxx"}...