dd-trace-js icon indicating copy to clipboard operation
dd-trace-js copied to clipboard

add automatic heap snapshots to help debug memory issues

Open rochdev opened this issue 5 months ago • 4 comments

What does this PR do?

Add automatic heap snapshots to help debug memory issues.

Motivation

Sometimes heap dumps are needed to debug memory issues. This can be annoying for users as they have to manually add code to generate the heap dumps, and sometimes they are not generated correctly (wrong interval, only one sample, without garbage collection, etc). This change adds the ability to generate heap snapshots directly from dd-trace by simply passing an environment variable, making the whole process a lot easier.

Additional Notes

@BridgeAR I know you have been working on logic to write the heap snapshot manually so that we can also send it elsewhere, let me know if you want me to add it to this PR, otherwise I think it could be a future improvement.

I thought about using telemetry for this but I don't like the security implications of sending a full memory dump directly to us.

rochdev avatar Jun 20 '25 17:06 rochdev

Overall package size

Self size: 9.58 MB Deduped: 107.65 MB No deduping: 108.17 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 9.0.0 | 19.6 MB | 19.61 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.9.0 | 9.77 MB | 10.14 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.3 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.14.2 | 122.36 kB | 850.93 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.9 | 25.11 kB | 25.11 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

github-actions[bot] avatar Jun 20 '25 17:06 github-actions[bot]

Codecov Report

Attention: Patch coverage is 93.54839% with 2 lines in your changes missing coverage. Please review.

Project coverage is 81.86%. Comparing base (3cb6f65) to head (965e3c1). Report is 49 commits behind head on master.

Files with missing lines Patch % Lines
packages/dd-trace/src/heap_snapshots.js 96.00% 1 Missing :warning:
packages/dd-trace/src/proxy.js 50.00% 1 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5920      +/-   ##
==========================================
+ Coverage   79.51%   81.86%   +2.34%     
==========================================
  Files         476      477       +1     
  Lines       20302    19527     -775     
==========================================
- Hits        16144    15985     -159     
+ Misses       4158     3542     -616     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Jun 20 '25 17:06 codecov[bot]

Benchmarks

Benchmark execution time: 2025-07-03 15:37:08

Comparing candidate commit 965e3c15959d3f8d472bfb60219b2bbd06e5be61 in PR branch auto-heap-snapshots with baseline commit 3cb6f656ba82e8afcf8abc4c0e47fb78d67147b2 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1275 metrics, 48 unstable metrics.

pr-commenter[bot] avatar Jun 20 '25 17:06 pr-commenter[bot]

Datadog Report

Branch report: auto-heap-snapshots Commit report: 39f4cfb Test service: dd-trace-js-integration-tests

:white_check_mark: 0 Failed, 1260 Passed, 0 Skipped, 14m 57.03s Total Time

LGTM with @IlyasShabi s comment addressed

You are still blocking the PR 😅

rochdev avatar Jul 03 '25 19:07 rochdev