client_golang
client_golang copied to clipboard
prometheus
Provide statsd bridge
I'm not sure how well the semantics of Prometheus and statsd will work together, but if there is a way, we would certainly welcome a PR. (Or in other words: I don't have plans to do it myself.)
I think it could work by exposing everything as a statsd gauge. It's unclear exactly which types are accepted, the docs hint uint64, but there's other indications that floats are okay.
In letsencrypt/boulder, we implemented our own statsd bridge interface: https://github.com/letsencrypt/boulder/blob/master/metrics/scope.go#L10-L23. Note: we have since moved more fully off statsd, and so removed the error return type. In order to have strict compatibility with the go-statsd-client API, you'd need to add back an error return type to most of these methods. Also, the interface only defines a subset of the go-statsd-client methods, but it's a popular subset.
The main concepts: When a new stat name is observed, automatically register it with the Prometheus client if it hasn't been seen before, and keep a cache of such registrations. Try statsd counters as counters, gauges as gauges, and Timing and TimingDuration become Prometheus summaries.
If you'd like to take the time to abstract that into a more generalized bridge, I think that would be great!
In the Prometheus context, a bridge is something that a Prometheus client outputs to. So a statsd bridge would talk to a statsd.
Is the idea to have a separate process that consumes from Prometheues (Go or otherwise) metrics, buffers, computes deltas for counters and pushes them to statsd? Or to make existing Prometheus client Go API more versatile, and be able to somehow do that in process? I am not very keen about the second one, as it adds memory and dependencies for all users of Prometheus, and potentially significant overhead (especially if you decide to push every update to Statsd on counter's Inc() method, even adding a if branch or additional locks to Inc() method to check if or not to use statsd, could be a huge bottleneck).
Details have to be seen, but the basic idea is to provide a package like https://godoc.org/github.com/prometheus/client_golang/prometheus/graphite .
In short: You have already instrumented your software with the client_golang/prometheus package. You can use the promhttp package for HTTP exposition of the Prometheus format. If you now want to send the metrics to graphite instead, you use the graphite package mentioned above. The idea here is to provide a similar package that allows to send those metrics to statsd (probably everything as statsd gauge, as Brian said).
This means that nothing will change if you don't use that package,. (i.e. no added memory or dependencies for all users of Prometheus).