kafka-ui icon indicating copy to clipboard operation
kafka-ui copied to clipboard

Invalid value for "delete.retention.ms" results in Kafka Broker crash

Open ChSchott opened this issue 3 years ago β€’ 11 comments

Entering an invalid value for the "delete.retention.ms" parameter causes the Kafka Broker to crash.

Log from the Kafka Broker: org.apache.kafka.common.config.ConfigException: Invalid value -1 for configuration delete.retention.ms: Value must be at least 0

ChSchott avatar Aug 29 '22 16:08 ChSchott

Hello there ChSchott! πŸ‘‹

Thank you and congratulations πŸŽ‰ for opening your very first issue in this project! πŸ’–

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. πŸ‘€

github-actions[bot] avatar Aug 29 '22 16:08 github-actions[bot]

Hey, thanks for reaching out. We'll take a look.

Haarolean avatar Aug 29 '22 22:08 Haarolean

@Haarolean Can you please assign this to me ?

shubhwip avatar Aug 30 '22 04:08 shubhwip

@ChSchott How did you specify "delete.retention.ms" property ? through kafka-ui application or other ways ? Can you specify steps for reproducing the issue ?

shubhwip avatar Aug 30 '22 08:08 shubhwip

@Haarolean If you have an idea how this could be reproduced. Please comment so that i can look into it further.

shubhwip avatar Sep 03 '22 15:09 shubhwip

@ChSchott there is a bold and certain reason why we do have issue templates set in place. The one for a bug report (which you used it seems but decided not to follow one) contains an important block named "Steps to Reproduce". Please provide the steps.

Haarolean avatar Sep 03 '22 16:09 Haarolean

This issue has been automatically marked as stale because no requested feedback has been provided. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Sep 11 '22 02:09 github-actions[bot]

@Haarolean To reproduce this error you can use the attached docker-compose file. This docker-compose file will start a single Kafka instance in Kraft mode (without Zookeeper). Dockerfiles.zip

Once the kafka and kafka-ui instance are running:

  1. Open your Browser and navigate to localhost:8080

  2. Click on "Add a Topic" as shown below Add_Topic

  3. Enter the following custom topic parameters Configure_Topic

  4. Click Submit

  5. After clicking Submit you while see an error message in the bottom right Submit_Topic

  6. Now navigate back to the dashboard. Make sure to refresh the Page. The Dashboard shows one kafka instance that is offline. Reload_Dashboard

  7. Now open a terminal and retrieve the the logs from Kafka: docker logs kafka Kafka_Logs

The kafka instance is not able to recover itself for some reason. The only solution i found to resolve this issue is to delete all kafka docker volumes and thereafter restart the kafka container.

Any questions? Let me know. Cheers.

ChSchott avatar Sep 13 '22 11:09 ChSchott

@ChSchott cool, thanks. @shubhwip kraft mode part might be important here. Still interested to take a look?

Haarolean avatar Sep 13 '22 15:09 Haarolean

@ChSchott cool, thanks. @shubhwip kraft mode part might be important here. Still interested to take a look?

Yeah Sure :)

shubhwip avatar Sep 13 '22 15:09 shubhwip

Thanks @ChSchott.

@Haarolean I Reproduced this issue. basically as per kafka documentation, -1 is invalid value for delete.retention.ms so this invalid value is not handled properly on kafka side thus resulting in broker crash. https://docs.confluent.io/platform/current/installation/configuration/topic-configs.html#topicconfigs_delete.retention.ms

We can have below ways to deal with this problem.

  • We don't do anything because users should be aware about all these properties by seeking kafka confluent documentation :)
  • we can show valid values next to each custom parameters to make sure users are informed and aware before providing values.
  • Enforce valid values for each type of custom parameters in kafka-ui to avoid any invalid values. Currently we’d generalised same input type for all types of custom parameters. But if we want to enforce this then we need to have different types of input for different params. For e.g. property [compression.type](https://docs.confluent.io/platform/current/installation/configuration/topic-configs.html#topicconfigs_compression.type) has following valid values [uncompressed, zstd, lz4, snappy, gzip, producer] so a dropdown would be required so that end user won’t be able to do any mistake. Other property for e.g. [delete.retention.ms] would require input type number with min and max values as set.

shubhwip avatar Sep 14 '22 10:09 shubhwip

@Haarolean Did you get a chance to look at my above comment :) ?

shubhwip avatar Oct 10 '22 06:10 shubhwip

@Haarolean Did you get a chance to look at my above comment :) ?

Yeaah I did, just left it without an answer.

Let's leave it as is for now, other options are too much work and not ideal.

Haarolean avatar Oct 14 '22 09:10 Haarolean

Closing for now as we can't know all the possible values for all the possible properties.

Haarolean avatar Oct 14 '22 09:10 Haarolean