Dana Powers comments

Results 48 comments of


                                            Dana Powers

Add context manager support to Consumers and producers

fwiw, there is an old (unsupported) context manager in kafka.context that was designed to manage consumer offsets w/ SimpleConsumer.

Add context manager support to Consumers and producers

Another alternative would be: ``` import contextlib from kafka import KafkaProducer with contextlib.closing(KafkaProducer(bootstrap_servers='localhost:1234')) as producer: for _ in range(100): producer.send('foobar', b'some_message_bytes') ```

Console commands

perhaps. but omar was a close friend who passed away and I have a hard time closing this for personal reasons.

Console commands

Peace brother - merged #2560 w/ console interfaces.

Support for KIP-429: Incremental rebalancing

Link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-429%3A+Kafka+Consumer+Incremental+Rebalance+Protocol

`KafkaConsumer.poll()` does not poll messages with default value for `timeout_ms`

Do you have logs with more details? The expected behavior is that there is no IO blocking within poll() that lasts longer than the timeout value. If timeout is 0...

KIP-392: Include client.rack property in Kafka Consumer

https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica

How to dynamically adjust the number of partitions?

Currently requires a `NewPartitions` object instance. This should work: ``` from kafka.admin.new_partitions import NewPartitions # Increase the partition count for 'foo_topic' to 6. admin_client.create_partitions({'foo_topic': NewPartitions(total_count=6)}) ```