kafka-ui icon indicating copy to clipboard operation
kafka-ui copied to clipboard

Allow users to specify a custom topic parameter (ie., `local.retention.bytes`)

Open tobernguyen opened this issue 1 year ago • 3 comments

Issue submitter TODO list

  • [X] I've searched for an already existing issues here
  • [X] I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?

We are using MSK (AWS Kafka) with Tiered storage. In order to use tiered storage, we need to set this config during topic creation, which is not currently supported by kafka-ui

remote.storage.enable = true
local.retention.ms = 600000

Describe the feature you're interested in

I'm thinking of 2 following options:

  1. Add 2 new custom parameters:
  • remote.storage.enable
  • local.retention.ms
  • local.retention.bytes
  1. Allow the user to specify a "real" custom parameter. Basically, we allow users to enter whatever param name and value they want. I think this will make kafka-ui future-proof because there will be more topic configs in the future.

Describe alternatives you've considered

No response

Version you're running

56fa824

Additional context

Related code sections:

  • UI code to define available custom params: https://github.com/provectus/kafka-ui/blob/53a6553765a806eda9905c43bfcfe09da6812035/kafka-ui-react-app/src/lib/constants.ts#L22
  • e2e test code for the UI: https://github.com/provectus/kafka-ui/blob/53a6553765a806eda9905c43bfcfe09da6812035/kafka-ui-e2e-checks/src/main/java/com/provectus/kafka/ui/pages/topics/enums/CustomParameterType.java
  • API Code: https://github.com/provectus/kafka-ui/blob/53a6553765a806eda9905c43bfcfe09da6812035/kafka-ui-api/src/main/java/com/provectus/kafka/ui/model/InternalTopic.java

tobernguyen avatar Sep 18 '23 23:09 tobernguyen

Hello there tobernguyen! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

github-actions[bot] avatar Sep 18 '23 23:09 github-actions[bot]

I'm thinking of 2 following options:

  1. Add 2 new custom parameters:
  • remote.storage.enable
  • local.retention.ms
  • local.retention.bytes
  1. Allow the user to specify a "real" custom parameter. Basically, we allow users to enter whatever param name and value they want. I think this will make kafka-ui future-proof because there will be more topic configs in the future.

I have implemented the first option for my own use. Please let me know if you want me to open a PR to contribute back to the upstream: https://github.com/provectus/kafka-ui/compare/master...tobernguyen:kafka-ui:add_topic_params_msk_tiered_storage

tobernguyen avatar Sep 19 '23 21:09 tobernguyen

@tobernguyen Please make a PR 👍 This is the only UI tool that supports MSK IAM Auth + Glue schema registry with Avro

jelledv avatar Oct 17 '23 09:10 jelledv