heroku-datadog-drain-golang
heroku-datadog-drain-golang copied to clipboard
Send memory and load stats using default datadog event strings
Right now, this repo is sending things like:
prefix.app.metric.load.avg.1m:0.010000|g|#tag1,tag2
According to my (extremely limited) understanding, the default load events should be something like
system.load.1
or system.load.15
. These metrics are automatically created by datadog and shown in the infrastructure overview. It seems this package should parse and send the runtime metrics using the events expected by datadog by default.
Of course, I could be missing something. Any feedback would be appreciated.
@tylerb It could very well be the case, do you have some pointers to datadog documentation so we can have a look on how it works etc. as those load metrics from heroku are already aggregated and further processing might be the wrong thing to do.
If you are using datadog agent directly is better send load as histogram. For heroku you can use gauge to skip calculating histograms from value. You don't need average of average that make more smoothly outputs but completely non relevant.
I can't find any documentation about this specifically, but here's a bit more info.
Firstly, from the infrastructure list we get this output:

You can see that everything that is not heroku is showing various metrics that datadog expects by default.
Secondly, when you click on one of the hosts and look at the default metrics, you see nothing populated (as expected). Clicking the gear and selecting metric info shows this:

It looks like some/most of these metrics are being sent to datadog, just in the wrong format.
Thanks for clarification, now looking at it it seems that we are actually using the aggregates directly in our own graph and we are not trying to utilize this datadog functionality so we have never encountered it. Maybe it did not exist when we were solving it for ourselves. We'll have a look on what can be done.