Add memory profiling support

[dferramosi]

Is your feature request related to a problem? Please describe. The current implementation does not support memory profiling. There is an existing gem that supports memory profiling in a way that looks compatible with how samples are collected, which could provide a way to get started. I saw that ruby-prof is already in use for process time, which also has memory profiling support.

Describe the goal of the feature Provide memory profiling in ruby to help diagnose big/bloated endpoints and/or memory related bugs.

Describe alternatives you've considered Setting up two APMs is not great, and as we grow, memory tracing is an increasingly important issue

Additional context There are other APMs that support this including New Relic

[ivoanjo]

Hey Dominic, thanks for reaching out!

Memory profiles are definitely on our radar -- we hope to have something for profiling allocations to announce soon-ish, and having something for heap (or retained, as memory_profiler calls it) is also something we hope to look into :)

[voiski]

Any updates on this? I'm also interested in having it.

[ivoanjo]

I know we've been saying soon-ish for a while on this one. It's still being worked on, we even have an extra team member helping with this now :)

[ohookins]

@ivoanjo any ETA to releasing this?

[ivoanjo]

Yes! We actually released the first version as part as https://github.com/DataDog/dd-trace-rb/releases/tag/v1.19.0 -- you can start using it today!

This first version is marked as experimental -- we're making quick progress and expect it to hit beta and then GA soon.

[ohookins]

@ivoanjo thanks! I will try it out today! Sounds like this issue can be closed then.

[ohookins]

@ivoanjo where do you want feedback on the new feature?

[ivoanjo]

@ivoanjo thanks! I will try it out today! Sounds like this issue can be closed then.

I've been planning to close this ticket once we declare it as stable. Soon! :smile:

@ivoanjo where do you want feedback on the new feature?

Please do open a new ticket, so we can track and fix individually :)

[ivoanjo]

Hey folks! With the release of dd-trace-rb 1.21.1, we're officially marking allocation profiling as beta.

We've been doing lots of validation and performance improvements, and are quite happy with the result. Our plan is to mark it as stable once we get enough positive feedback from testers as well.

We also have heap profiling, still in alpha that you can also try out -- we've now shifted our focus to performance improvements and validations of this feature.

TL;DR:

1. You can enable allocation profiling with DD_PROFILING_ALLOCATION_ENABLED=true env variable or

Datadog.configure do |c|
  # ... existing configuration ...
  c.profiling.allocation_enabled = true
end

2. You can enable heap profiling with DD_PROFILING_EXPERIMENTAL_HEAP_ENABLED=true env variable or

Datadog.configure do |c|
  # ... existing configuration ...
  c.profiling.advanced.experimental_heap_enabled = true
end

(Note: heap builds atop allocation, so you'll need to enable both)

[ivoanjo]

Please do let us know any feedback you have after trying this out /cc @dferramosi @ctrombley @tsub @alemr-roo @zarakay @voiski @ohookins @rafal42 @sushmithaeshwar ;)

[dferramosi]

Please do let us know any feedback you have after trying this out

Just rolled this out yesterday, getting good feedback so far from the team!

[ivoanjo]

Since this is already supported in the latest versions of the library, going ahead and closing this ticket. We're still very interested on any feedback y'all have on these features, so please do get in touch :)