Slow Kafka loading and high CPU utilization after upgrading librdkafka from 0.11.6 to 2.4.0

[lazern]

Our app was using 0.11.6 librdkafka earlier to load data from kafka, recently we have upgraded the version of librdkafka to 2.4.0, after that we can see there is a significant degradation in performance.And also seeing high CPU utilization.

As a comparison, in 0.11.6 almost 5000 messages were consumed in 40 seconds where as 2.4.0 loading the same number of messages took 1 minutes 40 seconds.

We are using the below config, and not setting anything else.

queued.min.messages=10000 fetch.message.max.bytes=(24 * 1024 * 1024) message.max.bytes=(24 * 1024 * 1024) socket.max.fails=1 enable.partition.eof=true

same configs are in our app across version 0.11.6 and 2.4.0. Do you think if any change in config can help to resolve this situation ?