dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

WIP chore(ci_visibility): collect module-level line execution and imports

Open romainkomorndatadog opened this issue 8 months ago • 2 comments

NOTE: still very much a WIP

This explores changes necessary for coverage collection for the Intelligent Test Runner in order to be able to properly capture import-time dependencies on other modules.

To achieve this, the following are introduced:

  • optional instrumentation of import statements to capture that a given file imported a given module
  • optional capture of execution of module-level lines during coverage collection
  • optional reporting of import-time covered lines for modules traversed during regular coverage

The pytest usage of the internal coverage library is updated to use the collection of import-time lines, resulting in expanded coverage for all tests (or suites if using suite-level skipping).

Checklist

  • [ ] Change(s) are motivated and described in the PR description
  • [ ] Testing strategy is described if automated tests are not included in the PR
  • [ ] Risks are described (performance impact, potential for breakage, maintainability)
  • [ ] Change is maintainable (easy to change, telemetry, documentation)
  • [ ] Library release note guidelines are followed or label changelog/no-changelog is set
  • [ ] Documentation is included (in-code, generated user docs, public corp docs)
  • [ ] Backport labels are set (if applicable)
  • [ ] If this PR changes the public interface, I've notified @DataDog/apm-tees.

Reviewer Checklist

  • [ ] Title is accurate
  • [ ] All changes are related to the pull request's stated goal
  • [ ] Description motivates each change
  • [ ] Avoids breaking API changes
  • [ ] Testing strategy adequately addresses listed risks
  • [ ] Change is maintainable (easy to change, telemetry, documentation)
  • [ ] Release note makes sense to a user of the library
  • [ ] Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • [ ] Backport labels are set in a manner that is consistent with the release branch maintenance policy

romainkomorndatadog avatar Jun 21 '24 12:06 romainkomorndatadog