opentelemetry-dotnet icon indicating copy to clipboard operation
opentelemetry-dotnet copied to clipboard

Prometheus Exporer Dupplicate Tag error_type

Open vinh3c opened this issue 1 year ago • 4 comments

Bug Report

  • OpenTelemetry.Exporter.Prometheus.AspNetCore 1.7.0-rc.1
  • net8.0

Symptom

=> Prometheus Pull Agent label name "error_type" is not unique: invalid sample

image

When aspnetcore host nginx, Request has a Application Exception, after that the request connection is closed => aspnet core add more than one error tag

Reproduce

var tagsFeature = context
      .Features
      .Get<IHttpMetricsTagsFeature>();
tagsFeature.Tags.Add("error.type", "System.InvalidOperationException")
tagsFeature.Tags.Add("error.type", "MysqlConnector.MySqlException")

vinh3c avatar Dec 27 '23 01:12 vinh3c

+1

madyar0v avatar Jan 15 '24 05:01 madyar0v

Neither the OTel Metrics API (i.e Meter/instruments from DS), nor the OTel Metrics SDK performs de-duplication, as that will cause perf issues. However, exporters can do de-dup, if the backend it is exporting to cannot deal with duplicates. We can treat this as a bug in Prometheus Exporter, and fix it. Also need to check if OTLP exporter should also provide a feature to dedup on export.

cijothomas avatar Jan 19 '24 15:01 cijothomas

I've experienced the same problem.

For a web application running under IIS on Windows Server, after adding a second HTTPS binding, HTTPS redirection middleware starts generating exceptions for HTTP requests which results in duplicate error_type="System.InvalidOperationException", error_type="System.InvalidOperationException" items.

AlexBAV avatar Mar 03 '24 18:03 AlexBAV

At first glance this looks very similar to a problem I have been encountering in conjunction with websockets. Imo the fact that AspNetCore itself duplicates this tag is something that should be fixed upstream, so I reported it there.

Now if a user themselves introduced tag duplication that would be a different story.

hacst avatar Apr 16 '24 12:04 hacst

Imo the fact that AspNetCore itself duplicates this tag is something that should be fixed upstream, so I reported it there.

Now if a user themselves introduced tag duplication that would be a different story.

Closing this issue as there is no action item for OpenTelemetry.Exporter.Prometheus.AspNetCore.

reyang avatar May 21 '24 20:05 reyang

I got this error too

imclint21 avatar Jul 01 '24 20:07 imclint21

Imo the fact that AspNetCore itself duplicates this tag is something that should be fixed upstream, so I reported it there. Now if a user themselves introduced tag duplication that would be a different story.

Closing this issue as there is no action item for OpenTelemetry.Exporter.Prometheus.AspNetCore.

@reyang your exporter is not valid for Prometheus, so why close this issue? image

imclint21 avatar Jul 02 '24 12:07 imclint21