Schema registry times out connecting to zookeeper

[eli-jordan]

I am trying out the schema registry, and attempting to connect the docker container to a zookeeper/kafka running on a VM. However, it times out when attempting to connect to zookeeper.

However, when I run the schema registry directly on my laptop (i.e. no docker) it works just fine with the same configuration.

Is there some other configuration/setting I need to make this setup work?

The error I'm getting when trying to run the docker container is the following

metric.reporters = []
    kafkastore.connection.url = mesos-master:2181
    avro.compatibility.level = backward
    debug = false
    shutdown.graceful.ms = 1000
    response.mediatype.preferred = [application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json]
    kafkastore.commit.interval.ms = -1
    response.mediatype.default = application/vnd.schemaregistry.v1+json
    kafkastore.topic = _schemas
    metrics.jmx.prefix = kafka.schema.registry
    access.control.allow.origin = 
    port = 8081
    request.logger.name = io.confluent.rest-utils.requests
    metrics.sample.window.ms = 30000
    kafkastore.zk.session.timeout.ms = 30000
    master.eligibility = true
    kafkastore.topic.replication.factor = 3
    kafkastore.timeout.ms = 500
    host.name = cf1d1b09e022
    schema.registry.zk.namespace = schema_registry
    kafkastore.init.timeout.ms = 60000
    metrics.num.samples = 2
 (io.confluent.kafka.schemaregistry.rest.SchemaRegistryConfig:135)
[2016-04-08 13:19:18,701] INFO Initialized the consumer offset to -1 (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread:86)
[2016-04-08 13:19:28,924] ERROR Server died unexpectedly:  (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:51)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1120)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:147)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:122)
    at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:89)
    at kafka.utils.ZkUtils$.apply(ZkUtils.scala:71)
    at kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:181)
    at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:129)
    at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:66)
    at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:69)
    at io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:93)
    at io.confluent.kafka.schemaregistry.storage.KafkaStore.<init>(KafkaStore.java:109)
    at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.<init>(KafkaSchemaRegistry.java:136)
    at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:53)
    at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:37)
    at io.confluent.rest.Application.createServer(Application.java:109)
    at io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain.main(SchemaRegistryMain.java:43)

[wenruij]

I have met the same problem with kafka, i just followed steps:

docker pull confluent/zookeeper
docker pull confluent/kafka
docker run --name zk-1 -e zk_id=1 -e zk_server.1=${myhost_ipv4}:2888:3888 -p 2181:2181 -p 2888:2888 -p 3888:3888 confluent/zookeeper
docker run --name kafka-1 -e KAFKA_BROKER_ID=1 -e KAFKA_ZOOKEEPER_CONNECT=${myhost_ipv4}:2181 -p 9092:9092 confluent/kafka

Then it casts error like:

[2016-04-14 03:00:55,066] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:82)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
    at kafka.Kafka$.main(Kafka.scala:46)
    at kafka.Kafka.main(Kafka.scala)
[2016-04-14 03:00:55,089] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
[2016-04-14 03:00:55,097] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer)
[2016-04-14 03:00:55,098] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:82)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
    at kafka.Kafka$.main(Kafka.scala:46)
    at kafka.Kafka.main(Kafka.scala)
[2016-04-14 03:00:55,099] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)

[eli-jordan]

Note I was able to fix this issue by running the docker image with --net="host". So, I'm assuming zookeeper needs to make reverse connections, and without that switch the wrong IP is advertised. Not sure if that is correct though.

[eli-jordan]

Update on my previous comment, this is true for the schema-registry image on docker hub. However, I built the images locally, and the same issue occurs even with --net="host". Maybe its related to some changes in confluent v2.0.1.

[leifg]

I run in the same issue when starting the registry image. Is there already a solution?

[neerajtiwar]

Any solutions? I am also facing same issue with schema-registry docker: Logs below. My Kafka and zookeeper are running on VMs and connectivity is open.

[main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x20162aa9a4f000c closed [main] ERROR io.confluent.admin.utils.cli.KafkaReadyCommand - Error while running kafka-ready. [main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x20162aa9a4f000c org.apache.kafka.common.errors.TimeoutException: Timed out waiting for Kafka to create /brokers/ids in Zookeeper. timeout (ms) = 40000