Timeout overflows when over 2047.48...

[witzatom]

Description

I found that whenever I set the timeout for the Consumer.poll method > 2047.4... it will start behaving incorrectly.

It looks like there is a overflow in the timeout where the timeout is going to be modulo 2**31 // 10^6 ~= 2147.483648

If I set the timeout to 4305 I will get a ~ 10 second timeout.

How to reproduce

prepare a consumer to connect to an empty topic

consumer = Consumer(**props)
start = time.time()
msg = consumer.poll(2048.0)
print(f"This should take many seconds {time.time - start}")

Versions: confluent-kafka==1.5.0 confluent_kafka.libversion(): ('1.4.4', 17040639)

Consumer config

- properties:
  - auto.offset.reset='earliest'
  - bootstrap.servers='kafka:9092'
  - enable.auto.commit=False
  - group.id='relay.testing'

OS: Ubuntu 16.04.5 LTS

Kafka version: tested against docker image bitnami/kafka:2.6.0

Checklist

Please provide the following information:

• [x] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
• [x] Apache Kafka broker version:
• [x] Client configuration: {...}
• [x] Operating system:
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[jliunyu]

v1.5.0 is an old version. Please try to upgrade to the latest version. If you still see this error in the latest version, please feel free to let us know.

[jliunyu]

v1.5.0 is an old version. Please try to upgrade to the latest version. If you still see this error in the latest version, please feel free to let us know.

Please ignore my last comment. We are looking into a fix for it.

[jliunyu]

After added some debug log, the librdkafka is able to receive 4305000ms, I think we need to fix this integer overflow from librdkafka side.

[nhaq-confluent]

Closing this issue since it was rooted in librdkafka

[nhaq-confluent]

Created librdkafka issue https://github.com/confluentinc/librdkafka/issues/4640