kubeless
kubeless copied to clipboard
kafkacat error connecting to kafka in consume mode; list mode works
Is this a BUG REPORT or FEATURE REQUEST?: BUG REPORT
What happened: Installed kafkacat to look into messages generated by minio (in configured topic)
kafkacat -b [kafka service ip]:9092 -L
--> This command works fine
_Metadata for all topics (from broker -1: 10.152.183.146:9092/bootstrap):
1 brokers:
broker 1001 at kafka-0.broker.kubeless.svc.cluster.local:9092
4 topics:
topic "_consumer_offsets" with 50 partitions:
partition 0, leader 1001, replicas: 1001, isrs: 1001
kafkacat -C -b _[kafka service ip]_:9092 -t minio
% ERROR: Local: Host resolution failure: kafka-0.broker.kubeless.svc.cluster.local:9092/1001: Failed to resolve 'kafka-0.broker.kubeless.svc.cluster.local:9092': Name or service not known
Managed to fix this by adding following line in /etc/hosts [kafka service ip] kafka-0.broker.kubeless.svc.cluster.local
What you expected to happen: It should work without need to modify /etc/hosts
How to reproduce it (as minimally and precisely as possible): Install kafkacat and try to consume messages for particular topic
Anything else we need to know?:
Environment:
-
Kubernetes version (use
kubectl version
): Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.2", GitCommit:"17c77c7898218073f14c8d573582e8d2313dc740", GitTreeState:"clean", BuildDate:"2018-10-24T06:54:59Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.2", GitCommit:"17c77c7898218073f14c8d573582e8d2313dc740", GitTreeState:"clean", BuildDate:"2018-10-24T06:43:59Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"} -
Kubeless version (use
kubeless version
): Kubeless version: v1.0.0 -
Cloud provider or physical cluster: microk8s channel=1.12/stable
Hi @bipinm,
I assume you are using kafkacat
from a pod or similar within a cluster? I assume that because you can access private IPs like 10.152.183.146
.
I think that your issue is that the Kafka service name (by default) is just kafka
so the URL would be: kafka.kubeless.svc.cluster.local:9092
. Note that the URL should be <service_name>.<service_namespace>.svc.cluster.local:<svc_port>
.
Hi @andresmgot,
i'm using kafkacat from host (Ubuntu 1810 in VMWarePlayer), not from within pods. Funny thing is that kafkacat -L works but -C throws error [kafka service ip] = 10.152.183.146
With microk8s i can directly access the pod IPs from host. I had one issue though - kafka pod could not resolve zookeeper.kubeless:2181. sudo iptables -P FORWARD ACCEPT
helped fix this issue.
I have not touched the pods, just started it following the guide https://kubeless.io/docs/pubsub-functions/. Pods and services are automatically created, so the issue must be in yaml definitions ?
I am not familiar with kafkacat
nor microk8s
but it seems that the issue is that it's not able to resolve services
from the host. Is that a supported feature? Why do you think that may be an issue with the yaml definitions?
That is correct, its not able to resolve service from host.
Found a similar issue, almost the same as my problem - https://github.com/Yolean/kubernetes-kafka/issues/99
kafkacat is able to find the broker and list the topics but not consume (-C option). Solution listed there vs mine is different, though mine would potentially work only on microk8s.
I see, let us know if there is something we could do in our side.
I faced with the same issue, after creating new record in /etc/hosts for the Continer ID binding to his private IP address it's solved /etc/hosts cat /etc/hosts 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6
The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts 172.0.17.2 4e0ccc2d7ed9 >> bind
nslookup 4e0ccc2d7ed9 Server: 127.0.0.53 Address: 127.0.0.53#53
Non-authoritative answer: Name: 4e0ccc2d7ed9 Address: 172.0.17.2