Does burrow have a hard dependency on zookeeper?

[wlisac]

I'm wondering if it's possible to use burrow in a way that it doesn't depend on having access to zookeeper.

It looks like there is a Consumer-Kafka that can consume offsets directly from Kafka.

However, it looks like burrow may depend on Zookeeper for other internal storage.

The [zookeeper] heading specifies a Zookeeper ensemble to use in order to store metadata for modules and provide locking between multiple copies of Burrow. This allows you to run multiple copies of Burrow with only one being allowed to execute the notifiers at any time.

It also looks like the code will always attempt to setup a Zookeeper coordinator:

https://github.com/linkedin/Burrow/blob/a449cc412c48231a2b7347cd790f82b8ed18faf2/core/burrow.go#L36-L44

Is there a way to disable requiring zookeeper for a single-instance deployment of burrow?

Thanks for the great library!

[foobaar]

I'm wondering if this is possible too. I want to setup burrow without a dependency on zookeeper. I have a single instance deployment of burrow.

[grcunha]

I'm also wondering the same since kafka moved the offset storage from zookeeper to kafka brokers, kafka-consumer does not need to directly communicate with zookeeper. For example we're using Kafka in SaaS mode in Aiven.io and we don't have access to zookeeper now

[Qixuanw]

Is there any possibility that we can have this feature? The zookeeper is inside my kafka cluster and I do not have access to that port.

[tkellen]

Got around this with the following (running a dummy zookeeper alongside burrow):

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: burrow
  name: burrow
spec:
  replicas: 1
  selector:
    matchLabels:
      app: burrow
  template:
    metadata:
      name: burrow
      labels:
        app: burrow
    spec:
      containers:
      - name: zookeeper
        ports:
        - containerPort: 2181
        image: confluentinc/cp-zookeeper:5.4.1
        env:
          - name: ZOOKEEPER_CLIENT_PORT
            value: "2181"
      - image: zipcar/burrow
        name: burrow
        ports:
        - containerPort: 8000
          name: http
        volumeMounts:
        - mountPath: /etc/burrow
          name: burrow-config
      volumes:
      - name: burrow-config
        configMap:
          name: burrow-config

[foobaar]

I did exactly what @tkellen did and configured burrow.yaml to use zookeeper on localhost deployment.yaml

spec:
  containers:
  - image: foo/zookeeper:3.5.5
    imagePullPolicy: Always
    name: monitoring-burrow-zookeeper
    ports:
    - containerPort: 2181
      name: client
      protocol: TCP
...

burrow.yaml

zookeeper:
  servers:
    - localhost:2181
...

[erkexzcx]

+1

[nsagnett]

Hello!

I just want to know if this topic is still in progress? An other possibility to store information without Zookeeper can be use Kafka topic as Schema-Registry for example?