confluent-kafka-python
confluent-kafka-python copied to clipboard
confluentinc
`log_cb` and `stats_cb` don't appear to log anything with either the `SerializingProducer` or `DeserializingConsumer`
Description
I might be doing something wrong, but it appears that the log_cb and stats_cb on the SerializingProducer and DeserializingConsumer don't seem to add any additional output.
How to reproduce
Use EOS configurations for SerializingProducer and DeserializingConsumer (not sure if relevant).
Use a logging instance called LOGGER (ours is used throughout our app), set to DEBUG.
Use the following callable:
def _print_stats(json_stats):
LOGGER.debug(f'PRINT STATS:\n{json_stats}')
Configure either a SerializingProducer and DeserializingConsumer with the included following (additional) settings in the configuration dictionary:
{
"statistics.interval.ms": 5000,
"log_cb": LOGGER,
"stats_cb": _print_stats
}
Does not produce any extra output on top of what we normally log. Things like on_delivery work just fine.
Checklist
Please provide the following information:
- confluent-kafka-python: 1.8.2
- Apache Kafka broker version: 2.X (not sure)
- Operating system: Pop! OS 21.04, python 3.8
i think log_cb parameter is logger.
stats_cb is passed through to the underlying client so should work ...
i'll hopefully officially stabalise the API soon, labelling as a reminder to look into this.
I have verified that the correct parameter to use is logger.
@mhowlett - I have seen one more issue where log_cb is used instead of logger. I feel the confusion arises because log_cb property is being used in librdkafka (https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) and logger in the python client and in the python documentation (https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#pythonclient-configuration), we point to the librdkafka configurations first. I think, we should update python documentation to point to python specific config first then refer to librdkafka config.
