[AdminClient] Getting segmentation fault for describe acls

[atul-raghuwanshi]

Description

We have a setup where we want to query all acls using admin client and then filter them. While using AdminClient().describe_acls(acl_filter) we are getting segmentation fault.

Note that we get these when number of acls are around 0.2 Million(2*10^5).

Also, When the number of acls are lesser for example 50K the same code is running fine and describe acls works as expected.

As internally it is invoking rdkafka code, not sure how to fix this and could not get a lot of logs as well. Is there any config which can help here. Earlier we were relying on zookeeper client to query all the acls and filter on regex but moving forward to KRaft we have to use AdminClient only and so a blocker for our migration.

How to reproduce

Have around 0.2 Million acls and call describe_acls(). We tried with both rdkafka/confluent-kafka-python version 1.9 and 2.5.3.

Checklist

Please provide the following information:

• [x] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): 1.9
• [x] Apache Kafka broker version: 3.5
• [ ] Client configuration: {...}
• [x] Operating system: linux
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[allamiro]

could you share some logs? wondering if there is any additional insights or specific error tip or message ?

[atul-raghuwanshi]

could you share some logs? wondering if there is any additional insights or specific error tip or message ?

@allamiro we did not get any relevant logs but just got "Segmentation fault (core dumped)". I will try to see if we can enable some debug logs to get better loginfo but this should be reproducible for number of acls around 0.2 Million.