dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

ci(internal): make flaky wrapping test a subprocess test

Open brettlangdon opened this issue 5 months ago • 3 comments

Checklist

  • [x] PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • [ ] Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

brettlangdon avatar Jun 27 '25 13:06 brettlangdon

CODEOWNERS have been resolved as:

tests/internal/test_wrapping.py                                         @DataDog/debugger-python @DataDog/apm-core-python

github-actions[bot] avatar Jun 27 '25 13:06 github-actions[bot]

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 285 ± 5 ms.

The average import time from base is: 283 ± 4 ms.

The import time difference between this PR and base is: 2.1 ± 0.2 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 1.245 ms (0.44%)
ddtrace 0.667 ms (0.23%)
ddtrace.internal._unpatched 0.027 ms (0.01%)
json 0.027 ms (0.01%)
json.decoder 0.027 ms (0.01%)
re 0.027 ms (0.01%)
enum 0.027 ms (0.01%)
types 0.027 ms (0.01%)
ddtrace.bootstrap.sitecustomize 0.578 ms (0.20%)
ddtrace.bootstrap.preload 0.578 ms (0.20%)
ddtrace.internal.remoteconfig.client 0.578 ms (0.20%)

github-actions[bot] avatar Jun 27 '25 14:06 github-actions[bot]

Benchmarks

Benchmark execution time: 2025-07-01 19:51:06

Comparing candidate commit eeabcd24a09af31ebd01ed59c7a710d91559380c in PR branch brettlangdon/fix.flaky.internal.test with baseline commit 5fb2a8af26edfab9d0b1cccd04174361aa481473 in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 546 metrics, 3 unstable metrics.

scenario:iastaspects-strip_aspect

  • 🟥 execution_time [+787.861ns; +863.553ns] or [+7.579%; +8.307%]

pr-commenter[bot] avatar Jun 27 '25 14:06 pr-commenter[bot]