Add Consumer.io_event_enable

[stephan-hof]

Hello,

this pull requests adds a method called io_event_enable to the Consumer. It simply wraps rd_kafka_queue_io_event_enable from librdkafka.

My goal is to make integration with async frameworks easier. Using it allows the async framework to be immediately notified when a new message arrived in the queue. Avoiding,

• the use of threads
• busy loops around Consumer.poll(timeout=0)

I'm not entirely sure about the linux_asyncio_consumer.py. Should it be part of the confluent-kafka repository or not? My thinking was to show how io_event_enable could be used.

Update: Now that I checked the issue tracker somebody already came up with the idea: https://github.com/confluentinc/confluent-kafka-python/issues/185#issuecomment-382232005 so consider this pull-request as an implementation of it.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[edenhill]

I'm not entirely sure about the linux_asyncio_consumer.py.

Having this as an example is good :+1:

[stephan-hof]

Many thanks for looking at this pull request. I made a commit trying to address the changes requested by the review. I'm wondering now what is the next step on my side. Should I mark the conversations as 'resolved'? Or should I simply wait for the next review.

[edenhill]

We're currently in release code freeze, but will revisit this PR in January when the upcoming release is out. Thank you

[wbarnha]

Any update on this PR? I'd like to use Confluent Kafka's asynchronous methods in https://github.com/faust-streaming/faust

[stephan-hof]

Thanks for the ping. In the hope to speed-up things I rebased the branch on latest master.

[wbarnha]

Bumping for posterity. Is the code freeze over @edenhill?

We would really like this functionality to be implemented to use in Faust and many other asynchronous Kafka applications!

[harubi]

It would be amazing to have :) @edenhill

[stephan-hof]

It appears to me that @edenhill is not very active in the last months (hope you are well).

@pranavrth I see that you keep this project alive. So is this pull request something you cold have a look at please? If you like I can make a push to solve the conflicts on CHANGELOG.md

[woile]

Is there any other maintainer who can take a look at this 🙏🏻 ? This is a huge win for async kafka