aiokafka
aiokafka copied to clipboard
Add metrics
- We have some research to do before. Here's the list of metrics http://kafka.apache.org/documentation.html#monitoring (correct if wrong). Select ones of interest, some don't have any meaning in
asynciocontext (like number of select calls). - kafka-python has an implementation for metrics already. It would be best to reuse it to not support too much code. Some measurements are needed before thou.
Metrics are still not even documented in kafka-python. I think we should wait until it becomes a stable API there before adding those...
For now I'll remove those from 0.2.0 milestone
It's not something too complicated, but requires some time. Basically we need to take the metrics aggregators present in kafka-python and call sensors for respectful events. Like self._sensors.records_fetch_lag.record(highwater - last_offset) after each fetch to record lag samples.
Any progress? Just noticed the consumer does not have the method .metrics()
Any updates on this? Having producer/consumers metrics for aiokafka seems like a super useful thing to have for debugging client side issues or investigating client vs server problems.