go-carbon icon indicating copy to clipboard operation
go-carbon copied to clipboard

Instrument go-carbon with Prometheus

Open gunnihinn opened this issue 6 years ago • 4 comments

Apart from being delicious, delicious betrayal, having Prometheus monitor go-carbon is useful when running the system at scale to avoid a "who watches the watchmen" situation.

I have some work towards doing this in a branch. I have basic system and Go runtime metrics already, along with some instrumentation of carbonserver.

This issue is here so that if people have strong feelings about doing this, we can talk about them now before we invest more time in this line of patches.

gunnihinn avatar Nov 15 '18 13:11 gunnihinn

I like the idea. It would also be nice to add opentracing support to the carbonserver (and to the carbonapi).

lomik avatar Nov 16 '18 18:11 lomik

It would also be nice to add opentracing support to the carbonserver (and to the carbonapi).

Sure. I've never actually worked on a system with tracing support, so I'd need some time to tinker with a toy example before doing the work, and I'm not sure when I'll have the cycles to do that. If anyone else feels inspired to add this support in the meantime, that'd be great.

gunnihinn avatar Nov 19 '18 08:11 gunnihinn

Apologies for piggybacking on this issue, it's unclear to me if this is already resolved given that 0.14.0 adds support for the Prometheus exporter. Anyhoo...

Are there any plans to expose all of the existing go-carbon metrics (carbon.agents.*) via the exporter? Or is the thinking that anyone that would be using Prometheus and go-carbon in the same company is already using Grafana and can just pull from both data sources?

obfuscurity avatar Sep 27 '19 15:09 obfuscurity

@obfuscurity The main reason was explained by the topic starter, we did not want to monitor our graphite stack with graphite. We did not set an ultimate goal to have a parity between Prometheus and graphite metrics. So coverage isn't full. Especially in those parts which we are not using.
We are slowly adding those which are most important for us, but I doubt that we will cover everything.

You are welcome to contribute, or may I suggest you to open a new issue with a list of metrics you are interested in. This might increase a chance that someone (who has a courage and some spare cycles) will implement it.

azhiltsov avatar Sep 27 '19 19:09 azhiltsov