dd-trace-js icon indicating copy to clipboard operation
dd-trace-js copied to clipboard

use sandbox for express plugin test

Open rochdev opened this issue 10 months ago • 5 comments

What does this PR do?

Use sandbox for express plugin test.

Motivation

This PR is trying a simple strategy for running plugin tests on the final artifact of dd-trace instead of on the local code.

rochdev avatar Jan 27 '25 14:01 rochdev

Overall package size

Self size: 8.53 MB Deduped: 94.93 MB No deduping: 95.45 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.4.0 | 29.44 MB | 29.44 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.5.0 | 9.8 MB | 10.17 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.6.1 | 2.59 MB | 2.73 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

github-actions[bot] avatar Jan 27 '25 14:01 github-actions[bot]

Benchmarks

Benchmark execution time: 2025-01-27 18:48:33

Comparing candidate commit 10aa914ad896e3a2c46f546801a4a3a80fa4a535 in PR branch plugin-test-sandbox with baseline commit bf28dddb5e231f8743b1fe0e64afa778f7a78411 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 909 metrics, 24 unstable metrics.

pr-commenter[bot] avatar Jan 27 '25 14:01 pr-commenter[bot]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 65.05%. Comparing base (29c26b0) to head (34797be). :warning: Report is 1103 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #5157       +/-   ##
===========================================
- Coverage   90.38%   65.05%   -25.33%     
===========================================
  Files         151       14      -137     
  Lines        5198      930     -4268     
===========================================
- Hits         4698      605     -4093     
+ Misses        500      325      -175     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Jan 27 '25 19:01 codecov[bot]

@rochdev is this something you would still like to work upon?

BridgeAR avatar May 28 '25 23:05 BridgeAR

is this something you would still like to work upon?

@BridgeAR Yes, eventually all tests should be using the sandbox similar to the integration tests. However, before that happens it needs to be streamlined. This hasn't been prioritized yet.

rochdev avatar Jun 02 '25 15:06 rochdev

I think we should get back to this, when we really need it.

Using the sandbox has an overhead and I do not remember issues around code having a negative impact on each other. If I am not mistaken, it might even detect such problems with the current setup just as much as it could hide issues.

BridgeAR avatar Sep 27 '25 09:09 BridgeAR

I am closing this for now to limit the surface of things we are currently working on. Please open this again when coming back to it.

BridgeAR avatar Nov 05 '25 13:11 BridgeAR