dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

[feature-request] allow configuration of endpoints to skip tracing for Flask

Open jfmyers9 opened this issue 4 years ago • 7 comments

It would be great to be able to configure the Flask integration to skip tracing on certain endpoints.

For example, we have two very busy endpoints that are used for liveness and health checks, and they currently generate a large amount of traces in DataDog. It would be great if there was an easy way to configure the Flask integration to skip tracing for these endpoints.

Thoughts?

jfmyers9 avatar Jun 16 '20 20:06 jfmyers9

Hi @jfmyers9!

You can use filters for this so that the spans are dropped.

Let me know if you have any issues with configuring it 🙂.

Kyle-Verhoog avatar Jun 17 '20 15:06 Kyle-Verhoog

Thanks for pointing that out. I'll give that a shot for our use case.

jfmyers9 avatar Jun 17 '20 16:06 jfmyers9

Hi @Kyle-Verhoog,

I tried using filters, and that did work somewhat for our use case, but there was some behavior that I found surprising.

To validate this in development, I'm using the LogWriter from ddtrace.internal.writer to see the traces that get outputted to stdout. When I configure the filter to block a specific endpoint, I see that traces will still originate from this endpoint.

For example, without the filter, I see a trace outputted loosely in the following structure:

flask.request: trace-id
  postgres.query
  postgres.connection.rollback
  ...

where the flask request is an entire trace with many spans underneath it.

When I add the filter I see the following for the same endpoint:

postgres.query: trace-id1
postgres.connection.rollback: trace-id2
...

The overarching Flask trace is no longer present, but the spans underneath it are broken into individual traces.

The behavior that I was expecting was to see no traces for this endpoint. Is my understanding of how filters work incorrect?

jfmyers9 avatar Jun 19 '20 20:06 jfmyers9

Hmmm 🤔 interesting. AFAICT it should be dropping the entire trace if one of the spans matches. However, I suspect that async might be being used somewhere which results in separate "traces" which this filter wouldn't handle :confused:. What library are you using for postgres? Might be helpful to get a pip freeze while we're at it so I can try to reproduce. 🙂

Kyle-Verhoog avatar Jun 19 '20 21:06 Kyle-Verhoog

I think these would be the relevant versions of pip packages involved in the above traces:

ddtrace==0.38.1
Flask==1.0
gunicorn==19.9.0
psycopg2==2.8.4

jfmyers9 avatar Jun 19 '20 21:06 jfmyers9

Oh, in the mean-time we also have the dropping functionality at the trace agent: https://github.com/DataDog/datadog-agent/blob/5b5f1230cee74be5133666f176b61bc980ec624f/pkg/config/config_template.yaml#L726-L730 that you could try as well.

Kyle-Verhoog avatar Jun 19 '20 21:06 Kyle-Verhoog

Thanks for the pointer. Will look into if that is an option.

jfmyers9 avatar Jun 22 '20 14:06 jfmyers9

I'm closing this because of its old age, lack of recent activity, and the fact that there was a possible workaround provided.

emmettbutler avatar Nov 03 '23 14:11 emmettbutler