dd-trace-rb icon indicating copy to clipboard operation
dd-trace-rb copied to clipboard

[PROF-9470] Align heap recorder cleanup with GC activity (second try)

Open ivoanjo opened this issue 4 months ago • 2 comments

Change log entry

Add setting to lower heap profiling memory use/latency by cleaning up young objects after Ruby GC

What does this PR do?

This PR adds a new mechanism to lower heap profiling overhead (memory and latency): the ability to clean young objects being tracked by the heap profiler after a Ruby GC cycle runs, rather than waiting for serialization time.

This mechanism is currently off by default and controlled by the setting c.profiling.advanced.heap_clean_after_gc_enabled / DD_PROFILING_HEAP_CLEAN_AFTER_GC_ENABLED

We plan to run a bit more validation before enabling it by default.

Motivation:

By doing these cleanups, we lower memory usage (we don't need to wait until until next serialization to notice an object was already collected), and latency, because there's less work to be done at serialization time.

Additional Notes:

This PR started from https://github.com/DataDog/dd-trace-rb/pull/3906; the main change from that earlier PR is that the heap recorder clean after GC behavior has been changed to be a best-effort mechanism (e.g. it's optional and not required for correctness).

How to test the change?

This change includes test coverage. I've also added a new benchmarking configuration to evaluate this new setting using our usual harness.

ivoanjo avatar Oct 23 '24 10:10 ivoanjo