How to remove Kafka Topic Config entry using API? (OpType.DELETE from kafka-config cli)

[ajanikow]

Versions

Sarama	Kafka	Go
v1.27.2	2.4.0	1.14

Configuration

Any kind of connection (SASL_PLAINTEXT, SASL_SSL, SSL), Kafka API version "2.4.0".

Logs

From Client application: EOF

From Server:

org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: ALTER_CONFIGS, apiVersion: 0, connectionId: <HIDDEN>
Caused by: java.lang.NullPointerException: value
        at java.util.Objects.requireNonNull(Objects.java:228)
        at org.apache.kafka.common.requests.AlterConfigsRequest$ConfigEntry.<init>(AlterConfigsRequest.java:92)
        at org.apache.kafka.common.requests.AlterConfigsRequest.<init>(AlterConfigsRequest.java:149)
        at org.apache.kafka.common.requests.AbstractRequest.parseRequest(AbstractRequest.java:213)
        at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:70)
        at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:96)
        at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:923)
        at kafka.network.Processor.$anonfun$processCompletedReceives$1$adapted(SocketServer.scala:903)
        at scala.collection.Iterator.foreach(Iterator.scala:941)
        at scala.collection.Iterator.foreach$(Iterator.scala:941)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
        at scala.collection.IterableLike.foreach(IterableLike.scala:74)
        at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
        at kafka.network.Processor.processCompletedReceives(SocketServer.scala:903)
        at kafka.network.Processor.run(SocketServer.scala:793)
        at java.lang.Thread.run(Thread.java:748)

Problem Description

Hello!

I want to remove configuration from Kafka Topic (in principal restore it to default value).

I tried to:

• set it to empty - validation fails
• set it to nil - error like above
• set it to default from Kafka - but option is not marked as default anymore

Is there any possibility to revert this value (restore to default) using Sarama?

Using Kafka CLI kafka-config I'm able to restore default value.

Best Regards, Adam.

[royantman]

It looks like Sarama is doing the right thing when encoding the request, the value is supposed to be NULLABLE_STRING. Kafka however pretty much ignores the protocol documented as they do this in their code:

https://github.com/apache/kafka/blob/aeeb7b2f9a9abe8f49543a2278757722e5974cb3/clients/src/main/java/org/apache/kafka/common/requests/AlterConfigsRequest.java#L50-L52

Now I'm curios as to how deleting configs via CLI works...

(Update) CLI admin client is using a different API (44, IncrementalAlterConfigs)

[ghost]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the master branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[stoeffel]

Is there any update on this issue? Does a workaround for removing config entries exist? Thanks ❤️

[ajanikow]

@stoeffel You can just dump current config, remove expected key from map, apply it again (we do it currently for Topics, but for broker Dynamic config it does not work)

[fengyinqiao]

When to support it in admin.go?@ajanikow @dnwe

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[dnwe]

Can this now be solved via https://github.com/IBM/sarama/pull/2088/ and the IncrementalAlterConfigsOperationDelete operation?

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.