dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

draft

Open wconti27 opened this issue 1 year ago • 2 comments

Checklist

  • [ ] Change(s) are motivated and described in the PR description
  • [ ] Testing strategy is described if automated tests are not included in the PR
  • [ ] Risks are described (performance impact, potential for breakage, maintainability)
  • [ ] Change is maintainable (easy to change, telemetry, documentation)
  • [ ] Library release note guidelines are followed or label changelog/no-changelog is set
  • [ ] Documentation is included (in-code, generated user docs, public corp docs)
  • [ ] Backport labels are set (if applicable)
  • [ ] If this PR changes the public interface, I've notified @DataDog/apm-tees.

Reviewer Checklist

  • [ ] Title is accurate
  • [ ] All changes are related to the pull request's stated goal
  • [ ] Description motivates each change
  • [ ] Avoids breaking API changes
  • [ ] Testing strategy adequately addresses listed risks
  • [ ] Change is maintainable (easy to change, telemetry, documentation)
  • [ ] Release note makes sense to a user of the library
  • [ ] Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • [ ] Backport labels are set in a manner that is consistent with the release branch maintenance policy

wconti27 avatar Jul 02 '24 13:07 wconti27

Datadog Report

Branch report: conti/incorporate-otel-propagation Commit report: 04db3fa Test service: dd-trace-py

:x: 167 Failed (0 Known Flaky), 120032 Passed, 52143 Skipped, 4h 14m 4.41s Total duration (3h 46m 40.44s time saved) :snowflake: 13 New Flaky

:x: Failed Tests (167)

This report shows up to 5 failed tests.

  • test_data_streams_sns_to_sqs - test.py - Details

    Expand for error
    ubprocess Test "python -m unittest tests.contrib.botocore.test.BotocoreTest.test_data_streams_sns_to_sqs" Failed (exit code 1):
    root/.pyenv/versions/3.11.3/lib/python3.11/socket.py:777: ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('127.0.0.1', 39082), raddr=('127.0.0.1', 9126)>
     self._sock = None
    esourceWarning: Enable tracemalloc to get the object allocation traceback
    
    =====================================================================
    RROR: test_data_streams_sns_to_sqs (tests.contrib.botocore.test.BotocoreTest.test_data_streams_sns_to_sqs)
    ---------------------------------------------------------------------
    raceback (most recent call last):
     File "/root/project/tests/contrib/botocore/test.py", line 1168, in test_data_streams_sns_to_sqs
    ..
    
  • test_data_streams_sns_to_sqs - test.py - Details

    Expand for error
    ubprocess Test "python -m unittest tests.contrib.botocore.test.BotocoreTest.test_data_streams_sns_to_sqs" Failed (exit code 1):
    root/.pyenv/versions/3.10.11/lib/python3.10/socket.py:776: ResourceWarning: unclosed <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 45226), raddr=('127.0.0.1', 9126)>
     self._sock = None
    esourceWarning: Enable tracemalloc to get the object allocation traceback
    
    =====================================================================
    RROR: test_data_streams_sns_to_sqs (tests.contrib.botocore.test.BotocoreTest)
    ---------------------------------------------------------------------
    raceback (most recent call last):
     File "/root/project/tests/contrib/botocore/test.py", line 1168, in test_data_streams_sns_to_sqs
    ..
    
  • test_data_streams_sns_to_sqs - test.py

  • test_data_streams_sns_to_sqs - test.py

  • test_data_streams_sns_to_sqs_raw_delivery - test.py - Details

    Expand for error
    ubprocess Test "python -m unittest tests.contrib.botocore.test.BotocoreTest.test_data_streams_sns_to_sqs_raw_delivery" Failed (exit code 1):
    root/.pyenv/versions/3.10.11/lib/python3.10/socket.py:776: ResourceWarning: unclosed <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 44230), raddr=('127.0.0.1', 9126)>
     self._sock = None
    esourceWarning: Enable tracemalloc to get the object allocation traceback
    
    =====================================================================
    RROR: test_data_streams_sns_to_sqs_raw_delivery (tests.contrib.botocore.test.BotocoreTest)
    oto doesn't currently handle raw delivery message handling quite correctly.
    ---------------------------------------------------------------------
    raceback (most recent call last):
    ..
    

New Flaky Tests (13)

  • test_cmdi - test_pygoat.py - Last Failure

    Expand for error
    TTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/start?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc05b44cd10>: Failed to establish a new connection: [Errno 111] Connection refused'))
    
  • test_cmdi - test_pygoat.py - Last Failure

    Expand for error
    TTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/start?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc79abf34c0>: Failed to establish a new connection: [Errno 111] Connection refused'))
    
  • test_nohttponly_cookie - test_pygoat.py - Last Failure

    Expand for error
    TTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/traces?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc7a31c38e0>: Failed to establish a new connection: [Errno 111] Connection refused'))
    
  • test_sqli - test_pygoat.py - Last Failure

    Expand for error
    TTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/start?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc79a9dc790>: Failed to establish a new connection: [Errno 111] Connection refused'))
    
  • test_sqli - test_pygoat.py - Last Failure

    Expand for error
    TTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/traces?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4600212550>: Failed to establish a new connection: [Errno 111] Connection refused'))
    

Benchmarks

Benchmark execution time: 2024-07-09 01:12:25

Comparing candidate commit 1e581cdb4ef16825bee59dfd79eeb67a98f50490 in PR branch conti/incorporate-otel-propagation with baseline commit 9c9b5a7e9d5977438cb98a6d86cf00b23f7fa3ff in branch main.

Found 0 performance improvements and 28 performance regressions! Performance is the same for 175 metrics, 9 unstable metrics.

scenario:httppropagationextract-all_styles_all_headers

  • 🟥 execution_time [+10.668µs; +10.947µs] or [+16.448%; +16.878%]

scenario:httppropagationextract-b3_headers

  • 🟥 execution_time [+7.062µs; +7.177µs] or [+92.461%; +93.959%]

scenario:httppropagationextract-b3_single_headers

  • 🟥 execution_time [+7.118µs; +7.203µs] or [+106.059%; +107.326%]

scenario:httppropagationextract-datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_match

  • 🟥 execution_time [+10.861µs; +11.152µs] or [+21.464%; +22.040%]

scenario:httppropagationextract-datadog_tracecontext_tracestate_propagated_on_trace_id_match

  • 🟥 execution_time [+10.776µs; +11.171µs] or [+19.775%; +20.500%]

scenario:httppropagationextract-full_t_id_datadog_headers

  • 🟥 execution_time [+8.189µs; +8.335µs] or [+51.785%; +52.708%]

scenario:httppropagationextract-invalid_priority_header

  • 🟥 execution_time [+6.982µs; +7.094µs] or [+143.259%; +145.544%]

scenario:httppropagationextract-invalid_span_id_header

  • 🟥 execution_time [+6.980µs; +7.078µs] or [+142.879%; +144.877%]

scenario:httppropagationextract-invalid_trace_id_header

  • 🟥 execution_time [+7.025µs; +7.127µs] or [+143.441%; +145.516%]

scenario:httppropagationextract-large_header_no_matches

  • 🟥 execution_time [+6.955µs; +7.134µs] or [+26.054%; +26.725%]

scenario:httppropagationextract-large_valid_headers_all

  • 🟥 execution_time [+6.930µs; +7.075µs] or [+24.864%; +25.383%]

scenario:httppropagationextract-medium_header_no_matches

  • 🟥 execution_time [+6.935µs; +7.048µs] or [+82.644%; +83.995%]

scenario:httppropagationextract-medium_valid_headers_all

  • 🟥 execution_time [+7.007µs; +7.131µs] or [+71.619%; +72.884%]

scenario:httppropagationextract-none_propagation_style

  • 🟥 execution_time [+6.911µs; +6.991µs] or [+194.864%; +197.133%]

scenario:httppropagationextract-tracecontext_headers

  • 🟥 execution_time [+9.344µs; +9.569µs] or [+38.909%; +39.846%]

scenario:httppropagationextract-valid_headers_all

  • 🟥 execution_time [+6.899µs; +6.988µs] or [+141.012%; +142.837%]

scenario:httppropagationextract-valid_headers_basic

  • 🟥 execution_time [+6.868µs; +6.976µs] or [+154.384%; +156.795%]

scenario:httppropagationextract-wsgi_invalid_priority_header

  • 🟥 execution_time [+6.967µs; +7.062µs] or [+143.673%; +145.633%]

scenario:httppropagationextract-wsgi_invalid_trace_id_header

  • 🟥 execution_time [+6.901µs; +7.004µs] or [+140.421%; +142.515%]

scenario:httppropagationextract-wsgi_large_header_no_matches

  • 🟥 execution_time [+6.976µs; +7.142µs] or [+26.005%; +26.623%]

scenario:httppropagationextract-wsgi_large_valid_headers_all

  • 🟥 execution_time [+6.996µs; +7.175µs] or [+25.070%; +25.712%]

scenario:httppropagationextract-wsgi_medium_header_no_matches

  • 🟥 execution_time [+7.009µs; +7.127µs] or [+83.940%; +85.354%]

scenario:httppropagationextract-wsgi_medium_valid_headers_all

  • 🟥 execution_time [+6.948µs; +7.060µs] or [+70.738%; +71.887%]

scenario:httppropagationextract-wsgi_valid_headers_all

  • 🟥 execution_time [+6.830µs; +6.935µs] or [+140.215%; +142.373%]

scenario:httppropagationextract-wsgi_valid_headers_basic

  • 🟥 execution_time [+7.024µs; +7.128µs] or [+157.617%; +159.934%]

scenario:httppropagationinject-with_all

  • 🟥 execution_time [+5.451µs; +5.666µs] or [+23.171%; +24.083%]

scenario:httppropagationinject-with_priority_and_origin

  • 🟥 execution_time [+5.311µs; +5.453µs] or [+29.958%; +30.757%]

scenario:httppropagationinject-with_sampling_priority

  • 🟥 execution_time [+4.637µs; +4.772µs] or [+35.883%; +36.929%]

pr-commenter[bot] avatar Jul 02 '24 16:07 pr-commenter[bot]

This pull request has been automatically closed after a period of inactivity. After this much time, it will likely be easier to open a new pull request with the same changes than to update this one from the base branch. Please comment or reopen if you think this pull request was closed in error.

github-actions[bot] avatar Oct 09 '24 00:10 github-actions[bot]