sclasen
(low priority) Coda Hale Metrics for topic offset, stream metrics
Although we are currently using: http://quantifind.com/KafkaOffsetMonitor/
...it would be ideal if support for Coda Hale Metrics was supported in akka-kafka, so we could track:
- offset status for each topic (e.g. ratio of current consumer offset to topic head)
- consumption rate for each topic
- latency of fetch vs commit for each batch for each topic
- histogram of batch sizes per topic
- etc.
http://metrics.codahale.com/
I think as long as the MetricRegistry name is a Typesafe Config variable used by both akka-kakfa and any app also using Coda Hale metrics, I believe we can rendezvous via the:
com.codahale.metrics.SharedMetricRegistries.getOrCreate(String name) method.
(Right now, we are intensively using CodaHale metrics in all our apps, and feeding, e.g.: Graphite, Riemann, etc. -- although the KafkaOffsetMonitor listed earlier is a nice tool, we like to have the metrics fully integrated into our standard monitoring and alerting pipeline, and Coda Hale Metrics gets us there)
If all this is clear as mud, just ping me for more detail.
