Producer silently swallows produce error if it does not have Write permissions

[kahartma]

Description

We have a producer that produces to a topic and does not have Write permissions on it. If I would produce to it, I would expect the produce call to throw an exception. Instead, librdkafka only logs PARTCNT|rdkafka#producer-3| [thrd:ssl://localhost:9090/]: Topic test-topic partition count changed from 3 to 0.

I do produce an on_delivery callback and also call producer.flush() and producer.poll() after the produce.

I found some older related issues https://github.com/confluentinc/confluent-kafka-python/issues/738 and https://github.com/confluentinc/confluent-kafka-python/issues/704 which are already closed with v1.6.0, will provide improved authentication error reporting. There is also this recently opened issue https://github.com/confluentinc/confluent-kafka-python/issues/1787 for a similar problem on the consumer.

How to reproduce

Initialize a producer without Write permissions and then run

def delivery_report(err: KafkaError, msg: Message) -> None:
  raise RuntimeError(err)

producer.produce("topic", "value", on_delivery=delivery_report)
producer.flush(timeout=1.0)
producer.poll(timeout=0)

You will observe the log message, but no error will be raised.

Checklist

Please provide the following information:

• [x] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): 2.6.0
• [x] Apache Kafka broker version: 3.7.1
• [x] Client configuration: {...}
• [ ] Operating system:
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [x] Critical issue