dd-trace-js icon indicating copy to clipboard operation
dd-trace-js copied to clipboard

ci: use official system tests GitHub Actions workflow

Open watson opened this issue 6 months ago • 5 comments

What does this PR do?

This runs all system tests instead of just a subset and tries to balance them out as much as possible, to get below the desired_execution_time.

Motivation

  • No need to maintain our own system tests workflow, when an official one exists
  • Better test coverage in PRs

Plugin Checklist

Additional Notes

watson avatar Jun 12 '25 05:06 watson

This stack of pull requests is managed by Graphite. Learn more about stacking.

watson avatar Jun 12 '25 05:06 watson

Overall package size

Self size: 9.61 MB Deduped: 106.12 MB No deduping: 106.64 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 8.5.2 | 19.33 MB | 19.34 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.8.2 | 9.56 MB | 9.93 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.3 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.14.0 | 120.58 kB | 841.68 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.9 | 25.11 kB | 25.11 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

github-actions[bot] avatar Jun 12 '25 05:06 github-actions[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 79.57%. Comparing base (55a80d5) to head (1ca7f34). Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5884      +/-   ##
==========================================
+ Coverage   79.16%   79.57%   +0.41%     
==========================================
  Files         455      457       +2     
  Lines       19930    19421     -509     
==========================================
- Hits        15778    15455     -323     
+ Misses       4152     3966     -186     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Jun 12 '25 05:06 codecov[bot]

Benchmarks

Benchmark execution time: 2025-06-27 11:25:52

Comparing candidate commit 1ca7f340bc64e1c8b4a53455d475c8ef41801b88 in PR branch watson/run-all-system-tests-on-prs with baseline commit 55a80d5b219592a977f1d5c64c5e703b9301e8fb in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1270 metrics, 53 unstable metrics.

pr-commenter[bot] avatar Jun 12 '25 06:06 pr-commenter[bot]

Datadog Report

Branch report: watson/run-all-system-tests-on-prs Commit report: 7bf5098 Test service: dd-trace-js-integration-tests

:white_check_mark: 0 Failed, 1261 Passed, 0 Skipped, 18m 53.26s Total Time

otherwise lgtm

simon-id avatar Jun 30 '25 10:06 simon-id

This is causing regressions compared to some optimizations we had in place before. I think this should either be reverted or we need to find how to apply those optimizations in the official system tests (which I believe should have been done first).

rochdev avatar Jul 02 '25 23:07 rochdev

@rochdev +1 if you observe regressions, what were the optimizations we had before ? you implemented them right ? cc @cbeauchesne

simon-id avatar Jul 07 '25 02:07 simon-id

@simon-id follow-up here : https://dd.slack.com/archives/C02LESJ9PQX/p1751885155925839?thread_ts=1751459086.562469&cid=C02LESJ9PQX

cbeauchesne avatar Jul 07 '25 12:07 cbeauchesne