[Bug report] Unable to connect to Apache Kafka

[justinmclean]

Version

main branch

Describe what's wrong

Running Apache Kafka locally and I've created a couple of topics:

./bin/kafka-topics.sh --list --bootstrap-server 192.168.1.44:9092
dummy
test

Created a catalog via the UI:

When I try to see teh topics it times out after about 30 seconds. Gravitino logs have these errors:

2024-05-06 01:15:20.709 WARN [kafka-admin-client-thread | gravitino.v1.uid2132248471925844124-metalake_demo.kafka] [org.apache.kafka.clients.NetworkClient.processDisconnection(NetworkClient.java:775)] - [AdminClient clientId=gravitino.v1.uid2132248471925844124-metalake_demo.kafka] Connection to node 1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.

2024-05-06 01:15:38.418 WARN [Gravitino-webserver-61] [com.datastrato.gravitino.server.web.rest.ExceptionHandlers$FilesetExceptionHandler.handle(ExceptionHandlers.java:277)] - Failed to operate fileset(s) operation [LIST] under schema [default], reason [Failed to list topics under the schema metalake_demo.kafka.default]
java.lang.RuntimeException: Failed to list topics under the schema metalake_demo.kafka.default

I'm not sure why it's referring to localhost there when I gave it another IP.

Error message and/or stacktrace

see above

How to reproduce

1. Follow the instructions here on running Kafka: https://kafka.apache.org/quickstart
2. Create some topics:

./bin/kafka-topics.sh --create --topic mytopic --bootstrap-server 192.168.1.44:9092 
``
3. Use REST interface or UI to list topics


### Additional context

_No response_

[mchades]

From the information currently available, this is related to the advertised.listeners of the Kafka cluster. Could you please check the config value in your Kafka and change it to your container's IP address

[justinmclean]

I'm not running Kafka in a container I'm running it locally also advertised.listeners is not set in server.properties and is commented out.

[justinmclean]

Adding that property with the correct IP address and restarting the Kafka server still gives the same error.

[justinmclean]

yes that is what I did and no difference

[justinmclean]

Using a new playground and setting advertised.listeners in server.properties fixed the issue.

[mchades]

Adding that property with the correct IP address and restarting the Kafka server still gives the same error.

It may be caused by the default config metadata.max.age.ms of the Kafka admin client, it can also be set by Gravitino catalog properties

[mchades]

I'm going to close this issue since it's a configuration issues with external services