dd-trace-go icon indicating copy to clipboard operation
dd-trace-go copied to clipboard

Use `runtime/metrics` instead of `runtime.ReadMemStats` and `runtime.ReadGCStats` for runtime metrics

Open ahmed-mez opened this issue 1 year ago • 0 comments

Opening this issue on behalf of @Gandem

runtime/metrics is supposed to be the new (and preferred) way of of exporting metrics from the Go runtime (added in Go 1.16). A couple of advantages:

  • It's more performant than alternatives: it does not involve a stop-the-world to be able to collect those metrics. Also, the metrics used by runtime/metrics are always collected in the go runtime whether you export them or not. So the overhead of actually exporting them is minimal (i.e. if we use runtime/metrics as a source, it could be reasonable to enable their collection by default, unless we're concerned about the number of metrics generated).
  • It contains metrics which are not available through other means such as the amount of GC scannable space, live heap/heap goal (rationale), distribution of goroutine latency (time goroutines have spent in the scheduler in a runnable state before actually running). And we can expect more metrics to be added here in the future.

One caveat is that runtime/metrics contains most of the metrics from ReadMemStats and ReadGcStats but not strictly all of the metrics (e.g. memory Sys, Lookups, LastGC).

@felixge's suggestion

we could add a new WithRuntimeMetricsV2 option for adding this feature. That would allow us to give our users the option to use the new runtime/metrics package exclusively without having to worry about backwards compatibility. This would also have the advantage that dd-go could consume this right away. And when we move to v2 for dd-trace-go, we just make this the default (we could still optionally offer the old version).

Bonus: We should explore enabling runtime metrics by default in dd-trace-go v2. Note that JVM metrics are enabled by default in the java tracer.

ahmed-mez avatar Sep 26 '23 08:09 ahmed-mez