Burrow
Burrow copied to clipboard
Does burrow have a hard dependency on zookeeper?
I'm wondering if it's possible to use burrow in a way that it doesn't depend on having access to zookeeper.
It looks like there is a Consumer-Kafka that can consume offsets directly from Kafka.
However, it looks like burrow may depend on Zookeeper for other internal storage.
The [zookeeper] heading specifies a Zookeeper ensemble to use in order to store metadata for modules and provide locking between multiple copies of Burrow. This allows you to run multiple copies of Burrow with only one being allowed to execute the notifiers at any time.
It also looks like the code will always attempt to setup a Zookeeper coordinator:
https://github.com/linkedin/Burrow/blob/a449cc412c48231a2b7347cd790f82b8ed18faf2/core/burrow.go#L36-L44
Is there a way to disable requiring zookeeper for a single-instance deployment of burrow?
Thanks for the great library!
I'm wondering if this is possible too. I want to setup burrow without a dependency on zookeeper. I have a single instance deployment of burrow.
I'm also wondering the same since kafka moved the offset storage from zookeeper to kafka brokers, kafka-consumer does not need to directly communicate with zookeeper. For example we're using Kafka in SaaS mode in Aiven.io and we don't have access to zookeeper now
Is there any possibility that we can have this feature? The zookeeper is inside my kafka cluster and I do not have access to that port.
Got around this with the following (running a dummy zookeeper alongside burrow):
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: burrow
name: burrow
spec:
replicas: 1
selector:
matchLabels:
app: burrow
template:
metadata:
name: burrow
labels:
app: burrow
spec:
containers:
- name: zookeeper
ports:
- containerPort: 2181
image: confluentinc/cp-zookeeper:5.4.1
env:
- name: ZOOKEEPER_CLIENT_PORT
value: "2181"
- image: zipcar/burrow
name: burrow
ports:
- containerPort: 8000
name: http
volumeMounts:
- mountPath: /etc/burrow
name: burrow-config
volumes:
- name: burrow-config
configMap:
name: burrow-config
I did exactly what @tkellen did and configured burrow.yaml to use zookeeper on localhost deployment.yaml
spec:
containers:
- image: foo/zookeeper:3.5.5
imagePullPolicy: Always
name: monitoring-burrow-zookeeper
ports:
- containerPort: 2181
name: client
protocol: TCP
...
burrow.yaml
zookeeper:
servers:
- localhost:2181
...
+1
Hello!
I just want to know if this topic is still in progress? An other possibility to store information without Zookeeper can be use Kafka topic as Schema-Registry for example?