kafka-docker
kafka-docker copied to clipboard
Sometimes kafka container is not starting properly
I have a very strange issue using the last version of this docker image. Given the following docker-compose:
version: '3'
services:
kafka:
image: wurstmeister/kafka:latest
ports:
- "9092:9092"
environment:
- KAFKA_ADVERTISED_HOST_NAME=kafka
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_CREATE_TOPICS=(my topic list)
depends_on:
- zookeeper
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
environment:
- KAFKA_ADVERTISED_HOST_NAME=zookeeper
sometimes (on travis) it fails with the following error. The strange thing is that doesn't fail all the times. Locally I was never able to reproduce it
kafka_1 | waiting for kafka to be ready
kafka_1 | [Configuring] 'advertised.port' in '/opt/kafka/config/server.properties'
kafka_1 | Excluding KAFKA_HOME from broker config
kafka_1 | [Configuring] 'advertised.host.name' in '/opt/kafka/config/server.properties'
kafka_1 | [Configuring] 'port' in '/opt/kafka/config/server.properties'
kafka_1 | [Configuring] 'broker.id' in '/opt/kafka/config/server.properties'
kafka_1 | Excluding KAFKA_VERSION from broker config
kafka_1 | [Configuring] 'zookeeper.connect' in '/opt/kafka/config/server.properties'
kafka_1 | [Configuring] 'log.dirs' in '/opt/kafka/config/server.properties'
kafka_1 | Error occurred during initialization of VM
kafka_1 | The flag -XX:+UseG1GC can not be combined with -XX:ParallelGCThreads=0
Thanks a lot!
Since today, I have exactly the same problem. But in my case it fails every time on one machine.
In case of ... I've got this error too on a LXC running on Proxmox. Thanks to https://github.com/itzg/docker-minecraft-server/issues/466 giving something to test :
Adding this to the proxmox lxc configuration file:
lxc.cgroup.devices.allow: c 195:* rwm
lxc.cgroup.devices.allow: c 235:* rwm
And it start now !
@slobberbone thank you for the hint. It turned out that it was also something to do with virtualization, which led to ParallelGCThreads=0. I fixed it by setting KAFKA_HEAP_OPTS: "-XX:ParallelGCThreads=1" in docker-compose.yml
I noticed it sometimes occurs when I have corrupted data. I simply down all the containers docker-compose down
and then prune the volumes docker volume prune
and then docker-compose up --build
and that fixes it.