aiokafka icon indicating copy to clipboard operation
aiokafka copied to clipboard

Add metrics

Open tvoinarovskyi opened this issue 9 years ago • 8 comments

  • We have some research to do before. Here's the list of metrics http://kafka.apache.org/documentation.html#monitoring (correct if wrong). Select ones of interest, some don't have any meaning in asyncio context (like number of select calls).
  • kafka-python has an implementation for metrics already. It would be best to reuse it to not support too much code. Some measurements are needed before thou.

tvoinarovskyi avatar May 02 '16 13:05 tvoinarovskyi

Metrics are still not even documented in kafka-python. I think we should wait until it becomes a stable API there before adding those... For now I'll remove those from 0.2.0 milestone

tvoinarovskyi avatar Nov 08 '16 14:11 tvoinarovskyi

It's not something too complicated, but requires some time. Basically we need to take the metrics aggregators present in kafka-python and call sensors for respectful events. Like self._sensors.records_fetch_lag.record(highwater - last_offset) after each fetch to record lag samples.

tvoinarovskyi avatar Apr 18 '17 09:04 tvoinarovskyi

Any progress? Just noticed the consumer does not have the method .metrics()

Corfucinas avatar Sep 06 '22 01:09 Corfucinas

Any updates on this? Having producer/consumers metrics for aiokafka seems like a super useful thing to have for debugging client side issues or investigating client vs server problems.

jigar-bhati avatar Oct 18 '23 16:10 jigar-bhati