kafka-rest
kafka-rest copied to clipboard
Set consumer setting max.poll.interval.ms to rest proxy setting of co…
…nsumer.timeout.ms
If consumer.timeout.ms has been set to a value greater than the default value of max.poll.interval.ms and a consumer has set auto.commit.enable=false then it is possible the kafka brokers will consider a consumer as failed and release its partition assignments, while the rest proxy maintains a consumer instance handle. This then leads to an exception on the next call to poll, commitSync, or similar.
This commit sets max.poll.interval.ms equal to consumer.timeout.ms to ensure that kafka will not consider the consumer failed until the rest proxy does as well.
Steps to reproduce the issue :
- Kafka version 0.10.1.0 or higher (where
max.poll.interval.msis introduced) - Set
consumer.timeout.msto a value greater than themax.poll.interval.msdefault of 5 minutes - Create a consumer via the rest proxy
- Sleep for a time greater than
max.poll.interval.msbut less thanconsumer.timeout.ms - Have consumer attempt to read a record or post an offset
It looks like @mmercedes hasn't signed our Contributor License Agreement, yet.
The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence. Wikipedia
You can read and sign our full Contributor License Agreement here.
Once you've signed reply with [clabot:check] to prove it.
Appreciation of efforts,
clabot
[clabot:check]
@confluentinc It looks like @mmercedes just signed our Contributor License Agreement. :+1:
Always at your service,
clabot
anyone able to take a look at this?
Bump
@stanislavkozlovski yes I have tested the issue locally and it resolved the issue I mentioned in my original comment. You can run these steps to reproduce the issue:
Steps to reproduce the issue :
- Kafka version 0.10.1.0 or higher (where max.poll.interval.ms is introduced)
- Set consumer.timeout.ms to a value greater than the max.poll.interval.ms default of 5 minutes
- Create a consumer via the rest proxy
- Sleep for a time greater than max.poll.interval.ms but less than consumer.timeout.ms
- Have consumer attempt to read a record or post an offset
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.