dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

feat(flask): trace streamed responses

Open mabdinur opened this issue 3 years ago • 9 comments

Description

In an instrumented flask application ddtrace does not include spans generated by a streamed response in the root flask.request span. Currently the flask.request span terminates before the first byte is streamed to the client.

To address this issue this proposal wraps flask applications in a wsgi middleware. The flask.request span produced by this middleware will encompass both receiving a request and sending the response to a client.

Screen Shot 2022-06-28 at 4 46 08 PM

Checklist

Motivation

Accurately measure the duration of flask request. Instrument streamed/buffered responses.

Design

Use the _BaseWSGIMiddleware introduced in this change: https://github.com/DataDog/dd-trace-py/pull/3907. By extending this base class we will wrap the flask request, the flask wsgi_app, and the response in a span.

Note - This is not a breaking change. The top level flask span flask.request will have the same tags and attributes (resource name, service name, error fields, etc). This PR introduces adds more visibility, it does not remove an existing functionality

Testing strategy

  1. create a flask app with an endpoint that returns a generator.
  2. run the flask app and send a request to the endpoint
  3. flush all spans to the agent
  4. ensure the following spans are generated:
  • flask.request
  • flask.application
  • flask.response

Reviewer Checklist

  • [ ] Title is accurate.
  • [ ] Description motivates each change.
  • [ ] No unnecessary changes were introduced in this PR.
  • [ ] PR cannot be broken up into smaller PRs.
  • [ ] Avoid breaking API changes unless absolutely necessary.
  • [ ] Tests provided or description of manual testing performed is included in the code or PR.
  • [ ] Release note has been added for fixes and features, or else changelog/no-changelog label added.
  • [ ] All relevant GitHub issues are correctly linked.
  • [ ] Backports are identified and tagged with Mergifyio.
  • [ ] Add to milestone.

mabdinur avatar Jun 16 '22 22:06 mabdinur

Codecov Report

Merging #3826 (5514fda) into munir/refactor-wsgi-middleware (57eef4f) will decrease coverage by 0.04%. The diff coverage is 0.00%.

@@                        Coverage Diff                         @@
##           munir/refactor-wsgi-middleware    #3826      +/-   ##
==================================================================
- Coverage                           79.09%   79.05%   -0.05%     
==================================================================
  Files                                 691      691              
  Lines                               53718    53746      +28     
==================================================================
  Hits                                42489    42489              
- Misses                              11229    11257      +28     
Impacted Files Coverage Δ
ddtrace/contrib/flask/patch.py 0.00% <0.00%> (ø)
tests/contrib/flask/test_errorhandler.py 0.00% <0.00%> (ø)
tests/contrib/flask/test_flask_appsec.py 0.00% <0.00%> (ø)
tests/contrib/flask/test_hooks.py 0.00% <0.00%> (ø)
tests/contrib/flask/test_request.py 0.00% <0.00%> (ø)
tests/contrib/flask/test_static.py 0.00% <0.00%> (ø)
tests/contrib/flask/test_views.py 0.00% <0.00%> (ø)
...ts/contrib/flask_autopatch/test_flask_autopatch.py 0.00% <0.00%> (ø)

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

codecov-commenter avatar Jul 06 '22 22:07 codecov-commenter

@mabdinur this pull request is now in conflict 😩

mergify[bot] avatar Jul 07 '22 20:07 mergify[bot]

Blocked by: https://github.com/DataDog/dd-trace-py/pull/3907

mabdinur avatar Jul 11 '22 18:07 mabdinur

@mabdinur this pull request is now in conflict 😩

mergify[bot] avatar Jul 12 '22 21:07 mergify[bot]

We can clean up the flask middleware code in a follow up change. Here's what I had in mind: 520d5beb9beb7390028e82b5eb7e1120fc10f44f. I would like to minimize the changes in this PR

mabdinur avatar Jul 15 '22 21:07 mabdinur

@mabdinur this pull request is now in conflict 😩

mergify[bot] avatar Jul 18 '22 17:07 mergify[bot]

@mabdinur this pull request is now in conflict 😩

mergify[bot] avatar Aug 02 '22 17:08 mergify[bot]

failing flask snapshots: https://app.circleci.com/pipelines/github/DataDog/dd-trace-py/20152/workflows/0e7c9f4e-80cb-4f3a-ab5c-1ea63afa7d8f/jobs/1364413.

Tests spot failing after this commit: https://app.circleci.com/pipelines/github/DataDog/dd-trace-py/20153/workflows/6eda8ea9-ea46-4d52-bc6e-6a89c49eaacd/jobs/1364513

mabdinur avatar Aug 09 '22 02:08 mabdinur

@mabdinur this pull request is now in conflict 😩

mergify[bot] avatar Aug 11 '22 11:08 mergify[bot]