rust-rdkafka icon indicating copy to clipboard operation
rust-rdkafka copied to clipboard

librdkafka returns some errors per partition

Open davidblewett opened this issue 4 months ago • 0 comments

While investigating something else, I noticed this:

/**
 * @brief Pause producing or consumption for the provided list of partitions.
 *
 * Success or error is returned per-partition \p err in the \p partitions list.
 *
 * @returns RD_KAFKA_RESP_ERR_NO_ERROR
 */
RD_EXPORT rd_kafka_resp_err_t
rd_kafka_pause_partitions(rd_kafka_t *rk,
                          rd_kafka_topic_partition_list_t *partitions);



/**
 * @brief Resume producing consumption for the provided list of partitions.
 *
 * Success or error is returned per-partition \p err in the \p partitions list.
 *
 * @returns RD_KAFKA_RESP_ERR_NO_ERROR
 */
RD_EXPORT rd_kafka_resp_err_t
rd_kafka_resume_partitions(rd_kafka_t *rk,
                           rd_kafka_topic_partition_list_t *partitions);

In other words, it always unconditionally returns no error from the API call but applications are supposed to check the passed in rd_kafka_topic_partition_list_t (our TopicPartitionList) for any errors. It's possible there are other APIs similar to this.

Unfortunately, we are not currently doing this:

  • https://github.com/fede1024/rust-rdkafka/blob/master/src/consumer/base_consumer.rs#L696
  • https://github.com/fede1024/rust-rdkafka/blob/master/src/consumer/base_consumer.rs#L706

We need to check for other cases that might work like this and update to loop over the TopicPartitionListElement.error() and return appropriately.

davidblewett avatar Feb 12 '24 16:02 davidblewett