[Bug] Support older kafka versions (v0.11.0)

[tzieleniewski]

Describe the bug There is no topic list displayed. But when I display the consumers list I am able to navigate to topic for this consumer group.

http://localhost:8080/ui/clusters/QA3/topics

There is a warning in logs

WARN  [parallel-2] o.h.v.i.p.j.JavaBeanExecutable: HV000254: Missing parameter metadata for TopicColumnsToSortDTO(String, int, String), which declares implicit or synthetic parameters. Automatic resolution of generic type information for method parameters may yield incorrect results if multiple parameters have the same erasure. To solve this, compile your code with the '-parameters' flag

Set up Version:0.3.3 Run through docker compose

 version: '3'
 services:
   magic:
     container_name: kafka-ui
     image: provectuslabs/kafka-ui
     ports:
       - "8080:8080"
     environment:
       - KAFKA_CLUSTERS_0_NAME=QA3
       - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=..
       - KAFKA_CLUSTERS_0_READONLY=true

Steps to Reproduce

docker-compose up

Expected behavior See list of topics.

Logs

kafka-ui  | 2022-02-04 12:43:05,887 INFO  [parallel-1] o.a.k.c.u.AppInfoParser: Kafka version: 2.8.0
kafka-ui  | 2022-02-04 12:43:05,887 INFO  [parallel-1] o.a.k.c.u.AppInfoParser: Kafka commitId: ebb1d6e21cc92130
kafka-ui  | 2022-02-04 12:43:05,888 INFO  [parallel-1] o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1643978585885
kafka-ui  | 2022-02-04 12:43:09,796 ERROR [kafka-admin-client-thread | adminclient-1] c.p.k.u.s.MetricsService: Failed to collect cluster QA3 info
kafka-ui  | org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeSynonyms at version 0
kafka-ui  | 2022-02-04 12:43:09,800 DEBUG [kafka-admin-client-thread | adminclient-1] c.p.k.u.s.ClustersMetricsScheduler: Metrics updated for cluster: QA3
kafka-ui  | 2022-02-04 12:43:35,671 DEBUG [parallel-1] c.p.k.u.s.ClustersMetricsScheduler: Start getting metrics for kafkaCluster: QA3
kafka-ui  | 2022-02-04 12:43:36,253 ERROR [kafka-admin-client-thread | adminclient-1] c.p.k.u.s.MetricsService: Failed to collect cluster QA3 info
kafka-ui  | org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_LOG_DIRS
kafka-ui  | 2022-02-04 12:43:36,253 DEBUG [kafka-admin-client-thread | adminclient-1] c.p.k.u.s.ClustersMetricsScheduler: Metrics updated for cluster: QA3
kafka-ui  | 2022-02-04 12:43:42,902 WARN  [parallel-2] o.h.v.i.p.j.JavaBeanExecutable: HV000254: Missing parameter metadata for TopicColumnsToSortDTO(String, int, String), which declares implicit or synthetic parameters. Automatic resolution of generic type information for method parameters may yield incorrect results if multiple parameters have the same erasure. To solve this, compile your code with the '-parameters' flag.

Additional context

[github-actions[bot]]

Hello there tzieleniewski! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

[Haarolean]

Hi,

Could you please share:

1. Which version are you using? There's a link to a commit in the upper left corner of the app
2. A screenshot with the topics page with enabled browser console (network tab)?

[tzieleniewski]

Howdy,

Please let me know if I can provide more info.

1. Which version are you using? There's a link to a commit in the upper left corner of the app https://github.com/provectus/kafka-ui/commit/38c4cf7

2. A screenshot with the topics page with enabled browser console (network tab)?

[Haarolean]

Thanks. Which kafka are you using? There are some problems with connecting to your cluster: kafka-ui | org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeSynonyms at version 0 kafka-ui | org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_LOG_DIRS

[tzieleniewski]

Thanks. Which kafka are you using? There are some problems with connecting to your cluster:

Howdy we are using 2.8.0

[Haarolean]

Thanks, we'll take a look into this.

[Haarolean]

Btw, how do you run the kafka itself? Docker/bare metal? If docker, which image name & version is it?

[tzieleniewski]

@Haarolean sorry but I had provided you with wrong version number. I've checked with other tool and it indicated that Kafka version is v0.11.0. Perhaps this is an issue. I got mislead by the version displayed in the application log.

[Haarolean]

@tzieleniewski oh wow! I wanted to ask you to double check the version, but noticed the same thing in logs as you did and decided not to. Yeaah, that's old as dirt. Would you consider upgrading?

[tzieleniewski]

@Haarolean would like that, but it does not depend on us, unfortunately.

thank you and take care

[Haarolean]

@tzieleniewski supporting such an old version would be a pretty huge load on development, if even possible (I dunno which APIs are present for this version). Unfortunately we don't have a bandwidth for this, but will consider it in the future.

Best wishes!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[rafaeldomi]

I think I have the same problem from this post. I've deployed a kafka cluster and I'm doing a test in which I stop one of the brokers. I can reproduce this issue. When I stop one broker, I can see that the only page (far now) that get errors is the [..]/ui/clusters/ClusterDev/topics. In the logs there is:

2022-09-20 18:55:25,425 ERROR [parallel-1] o.s.b.a.w.r.e.AbstractErrorWebExceptionHandler: [0bce61d8-79] 500 Server Error for HTTP GET "/api/clusters/ClusterDev/topics?showInternal=true&search=&orderBy=NAME&sortOrder=ASC" org.apache.kafka.common.errors.TimeoutException: Call(callName=metadata, deadlineMs=1663700125423, tries=591, nextAllowedTryMs=1663700125524) timed out at 1663700125424 after 591 attempt(s) Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: Error has been observed at the following site(s): *__checkpoint ⇢ Handler com.provectus.kafka.ui.controller.TopicsController#getTopics(String, Integer, Integer, Boolean, String, TopicColumnsToSortDTO, SortOrderDTO, ServerWebExchange) [DispatcherHandler] *__checkpoint ⇢ com.provectus.kafka.ui.config.ReadOnlyModeFilter [DefaultWebFilterChain] *__checkpoint ⇢ com.provectus.kafka.ui.config.CustomWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.authorization.AuthorizationWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.authorization.ExceptionTranslationWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.authentication.logout.LogoutWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.savedrequest.ServerRequestCacheWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.context.SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.authentication.AuthenticationWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.context.ReactorContextWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.header.HttpHeaderWriterWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ HTTP GET "/api/clusters/ClusterDev/topics?showInternal=true&search=&orderBy=NAME&sortOrder=ASC" [ExceptionHandlingWebHandler] Original Stack Trace: Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: metadata

I can confirm with kafkacat that the cluster is operational.

KafkaVersion: 3.2 (Deployed with strimzi) Kafka-ui: 0.4 (Deployed with helm)

[Haarolean]

@rafaeldomi hey, that's unrelated, you have a pretty new kafka. Please take a look at #2103, if that's the same issue you experience -- please leave a comment there, if not -- please raise a new one.

[newsbreak-tonglin]

@Haarolean Hi, do you have any plans about support older kafka version, like 0.11.0

[Haarolean]

@newsbreak-tonglin it's nowhere close to our priority list considering this would require major changes across the whole application's codebase.

[kukayiyi]

@newsbreak-tonglin it's nowhere close to our priority list considering this would require major changes across the whole application's codebase.

Hi, I have a similar problem with kafka2.11_1.0.1: org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeSynonyms at version 0 We need to clarify the usage rules of kafka-ui within our team. Is it possible to clarify the earliest version number of kafka currently supported by kafka-ui? Is it only available for kafka2 or above?

[Haarolean]

@kukayiyi hi, 2.x should work fine, yes.