opentelemetry-js
opentelemetry-js copied to clipboard
@opentelemetry/instrumentation-http should have an option to suppress tracing on ignored outgoing requests
Is your feature request related to a problem? Please describe.
As mentioned in https://github.com/open-telemetry/opentelemetry-js/discussions/3235, instrumentation-dns and instrumentation-net will create spans after an http span has been ignored. As a result, you're left with tls/tcp/dns spans that are essentially parentless (their parent is the http span's original parent before it was ignored). This adds a lot of noise in our traces, and there doesn't seem to be a good way to identify these spans and remove them.
Describe the solution you'd like
I'd love it if @opentelemetry-http had some sort of option that did a suppressTracing on this line https://github.com/open-telemetry/opentelemetry-js/blob/bb67268e17d466c2f24bc0106bcf7be0f101d688/experimental/packages/opentelemetry-instrumentation-http/src/http.ts#L635 similar to what was done previously in incoming spans here: https://github.com/open-telemetry/opentelemetry-js/issues/1831
Describe alternatives you've considered
I've looked into all available hooks on @opentelemetry/instrumentation-http and none of them can result in a suppressTracing behaviour to block the child spans from being sent.
I've tried to identify the net/dns spans in a Sampler, but there's no good way to know if they were spans part of a request i wanted to ignore or part of a request i wanted to keep (because it depends on parent information that's missing when i ignore an http span).
I've looked into using a SpanProcessor but that doesn't really work well to drop spans as far as I can tell - you can drop spans before export I'm sure, but done wrong it can mess up the tree of spans
It may be possible to climb the parents to the http call IF the http span is not ignored, and drop all the http, net, and dns spans that are associated with it in a span processor or sampler, BUT that will mean that propagation injection is on (since the http request is not ignored in order to generate the span) so that will cause children services to behave differently.