connect to docker-kafka from external client (using python)

[ugomariaguicciardini]

All,

Firstly let me apologize for the trivial question, but I am kafka newbie!

I am trying to run the docker-kafka container on a linux test server. I am able to run the usage-quick-start example succesfully.

The second step is to try to connect to kafka container from a separate machine using Kafka Python. I cannot figure out how to make this scenario working.

In particular i started docker-kafka container using a-more-complex-local-development-setup putting as KAFKA_ADVERTISED_HOST_NAME and ZOOKEEPER_IP the public IP of the (docker) host machine. Unfortunately the container crashes after few seconds, from the log is evident it is not able to connect to zookeeper. So i slightly changed the command line to:

docker run -d --name kafka --volume /home/cimino/kafka/ches/logs/:/logs --publish 9092:9092 --publish 7203:7203 --env KAFKA_ADVERTISED_HOST_NAME=docker-host-public-ip-here --link zookeeper:zookeeper ches/kafka

In this case the container is running, here is docker ps (PORTS column):

PORTS                                            
0.0.0.0:7203->7203/tcp, 0.0.0.0:9092->9092/tcp

But when i try to connect from an external client i got time out network connection. Here is the python comand line i am using (may this is OT, but i am reporting for completeness):

from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='docker-host-public-ip-here:9092')
producer.send('foobar', b'some_message_bytes')

What am I doing wrong?

Thanks UMG

[chengfish]

Same here. I am trying to use an external java client, and saw the similar issue.

[ches]

Sorry for delay in replying—I'll have to try to replicate this, but just to rule out a simple high-level possibility first: is there perhaps a firewall running on your Docker host machine that's getting in the way?

A simple check would be, from the client machine where you're trying to connect with the Python lib, does telnet <your Docker host address/IP> 9092 successfully connect? If not you may need to disable your firewall or allow the ports through.

[subdavis]

For anyone troubleshooting in the future, it really helps to run a simple TCP listener and try to connect.

On the host machine, nc -l 9092 while the docker image is not running. From the client where you are trying to connect telnet <docker host address> 9092

If you are unable to connect, the problem is not here.

[alejandrox1]

@ches this happens when running telnet:

$ telnet <docker host addrs> 9092
Trying <docker host addrs>...
Connected to <docker host addrs>.
Escape character is '^]'.
ruok
Connection closed by foreign host.

and the client still throws an error indicating no brokers are available.

Was there a fix for this issue?

[shaunc]

I'm getting the same thing... are you using a Mac? perhaps related to https://github.com/ches/docker-kafka/issues/26#issuecomment-292758707 ?