dd-trace-rb
dd-trace-rb copied to clipboard
[PROF-9470] Align heap recorder cleanup with GC activity (second try)
Change log entry
Add setting to lower heap profiling memory use/latency by cleaning up young objects after Ruby GC
What does this PR do?
This PR adds a new mechanism to lower heap profiling overhead (memory and latency): the ability to clean young objects being tracked by the heap profiler after a Ruby GC cycle runs, rather than waiting for serialization time.
This mechanism is currently off by default and controlled by the setting c.profiling.advanced.heap_clean_after_gc_enabled
/ DD_PROFILING_HEAP_CLEAN_AFTER_GC_ENABLED
We plan to run a bit more validation before enabling it by default.
Motivation:
By doing these cleanups, we lower memory usage (we don't need to wait until until next serialization to notice an object was already collected), and latency, because there's less work to be done at serialization time.
Additional Notes:
This PR started from https://github.com/DataDog/dd-trace-rb/pull/3906; the main change from that earlier PR is that the heap recorder clean after GC behavior has been changed to be a best-effort mechanism (e.g. it's optional and not required for correctness).
How to test the change?
This change includes test coverage. I've also added a new benchmarking configuration to evaluate this new setting using our usual harness.