jaeger
jaeger copied to clipboard
jaegertracing
Jaeger Collector cannot connect to Kerbrized Kafka with SASL SSL
Describe the bug
We are using kerbrized Kafka with SASL_SSL, Jaeger Collector cannot connect to Kerbrized Kafka
To Reproduce
docker run --net=abcd --name jaeger-collector -e SPAN_STORAGE_TYPE=kafka -p 14267:14267 -p 14268:14268 -p 9411:9411 -p 14250:14250 hboyina/jaegercollector:latest --kafka.producer.topic=jaeger-spans --kafka.producer.brokers=**:9093 --kafka.producer.authentication=kerberos --kafka.producer.kerberos.config-file=/etc/krb5.conf --kafka.producer.kerberos.keytab-file=/root/hboyina.keytab --kafka.producer.kerberos.use-keytab=true --kafka.producer.kerberos.username=hboyina --kafka.producer.kerberos.realm=.COM
Expected behavior Jaeger Collector should connect with Kafka , if the Kafka has SASL_SSL Authentication
Screenshots If applicable, add screenshots to help explain your problem.
Version (please complete the following information):
- OS: [e.g. Linux] Rhel
- Jaeger version: 1.22
- Deployment: Docker
What troubleshooting steps did you try? Try to follow https://www.jaegertracing.io/docs/latest/troubleshooting/ and describe how far you were able to progress and/or which steps did not work.
Additional context Add any other context about the problem here.
I have a related issue...
C:\Users\L0282162\Downloads>docker run -d --name jaeger-collector-quay -e SPAN_STORAGE_TYPE=kafka -e KAFKA_PRODUCER_BROKERS=pcdp-k01.bancogalicia.com.ar:9092,pcdp-k02.bancogalicia.com.ar:9092,pcdp-k03.bancogalicia.com.ar:9092,pcdp-k04.bancogalicia.com.ar:9092 -e KAFKA_PRODUCER_TOPIC=jaeger-spans -e KAFKA_PRODUCER_AUTHENTICATION=kerberos -e KAFKA_PRODUCER_KERBEROS_REALM=BGCMZ.BANCOGALICIA.COM.AR -e KAFKA_PRODUCER_KERBEROS_USERNAME=udoop22m@BGCMZ.BANCOGALICIA.COM.AR -e KAFKA_PRODUCER_KERBEROS_USE_KEYTAB=true -e LOG_LEVEL=debug -p 16686:16686 -p 14268:14268 -p 14250:14250 -v C:\Users\L0282162\Downloads\udoop22m.keytab:/etc/security/kafka.keytab quay.io/jaegertracing/jaeger-collector 97f5c6819456ac0d4825284ded7f15d7f75862dfc52a9f5a0d4b76623f7dc1fc
C:\Users\L0282162\Downloads>docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 97f5c6819456 quay.io/jaegertracing/jaeger-collector "/go/bin/collector-l…" About a minute ago Exited (1) About a minute ago jaeger-collector-quay
C:\Users\L0282162\Downloads>docker logs jaeger-collector-quay 2021/12/17 14:11:53 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined {"level":"info","ts":1639750313.6556678,"caller":"flags/service.go:117","msg":"Mounting metrics handler on admin server","route":"/metrics"} {"level":"info","ts":1639750313.6558056,"caller":"flags/service.go:123","msg":"Mounting expvar handler on admin server","route":"/debug/vars"} {"level":"info","ts":1639750313.65626,"caller":"flags/admin.go:104","msg":"Mounting health check on admin server","route":"/"} {"level":"info","ts":1639750313.6563847,"caller":"flags/admin.go:115","msg":"Starting admin HTTP server","http-addr":":14269"} {"level":"info","ts":1639750313.6564634,"caller":"flags/admin.go:96","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"} {"level":"info","ts":1639750313.6597028,"caller":"kafka/factory.go:69","msg":"Kafka factory","producer builder":{"Brokers":["pcdp-k01.bancogalicia.com.ar:9092","pcdp-k02.bancogalicia.com.ar:9092","pcdp-k03.bancogalicia.com.ar:9092","pcdp-k04.bancogalicia.com.ar:9092"],"RequiredAcks":1,"Compression":0,"CompressionLevel":0,"ProtocolVersion":"","BatchLinger":0,"BatchSize":0,"BatchMinMessages":0,"BatchMaxMessages":0,"Authentication":"kerberos","Kerberos":{"ServiceName":"kafka","Realm":"BGCMZ.BANCOGALICIA.COM.AR","UseKeyTab":true,"Username":"[email protected]","ConfigPath":"/etc/krb5.conf","KeyTabPath":"/etc/security/kafka.keytab"},"TLS":{"Enabled":false,"CAPath":"","CertPath":"","KeyPath":"","ServerName":"","ClientCAPath":"","SkipHostVerify":false},"PlainText":{"Username":"","Mechanism":"PLAIN"}},"topic":"jaeger-spans"} {"level":"fatal","ts":1639750314.6189425,"caller":"./main.go:80","msg":"Failed to init storage factory","error":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)","stacktrace":"main.main.func1\n\t./main.go:80\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/[email protected]/command.go:856\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/[email protected]/command.go:974\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/[email protected]/command.go:902\nmain.main\n\t./main.go:147\nruntime.main\n\truntime/proc.go:255"}
The cluster is working with other app. Any clue?
Thanks in advance.
I have a related issue...
C:\Users\L0282162\Downloads>docker run -d --name jaeger-collector-quay -e SPAN_STORAGE_TYPE=kafka -e KAFKA_PRODUCER_BROKERS=pcdp-k01.bancogalicia.com.ar:9092,pcdp-k02.bancogalicia.com.ar:9092,pcdp-k03.bancogalicia.com.ar:9092,pcdp-k04.bancogalicia.com.ar:9092 -e KAFKA_PRODUCER_TOPIC=jaeger-spans -e KAFKA_PRODUCER_AUTHENTICATION=kerberos -e KAFKA_PRODUCER_KERBEROS_REALM=BGCMZ.BANCOGALICIA.COM.AR -e KAFKA_PRODUCER_KERBEROS_USERNAME=[email protected] -e KAFKA_PRODUCER_KERBEROS_USE_KEYTAB=true -e LOG_LEVEL=debug -p 16686:16686 -p 14268:14268 -p 14250:14250 -v C:\Users\L0282162\Downloads\udoop22m.keytab:/etc/security/kafka.keytab quay.io/jaegertracing/jaeger-collector 97f5c6819456ac0d4825284ded7f15d7f75862dfc52a9f5a0d4b76623f7dc1fc
C:\Users\L0282162\Downloads>docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 97f5c6819456 quay.io/jaegertracing/jaeger-collector "/go/bin/collector-l…" About a minute ago Exited (1) About a minute ago jaeger-collector-quay
C:\Users\L0282162\Downloads>docker logs jaeger-collector-quay 2021/12/17 14:11:53 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined {"level":"info","ts":1639750313.6556678,"caller":"flags/service.go:117","msg":"Mounting metrics handler on admin server","route":"/metrics"} {"level":"info","ts":1639750313.6558056,"caller":"flags/service.go:123","msg":"Mounting expvar handler on admin server","route":"/debug/vars"} {"level":"info","ts":1639750313.65626,"caller":"flags/admin.go:104","msg":"Mounting health check on admin server","route":"/"} {"level":"info","ts":1639750313.6563847,"caller":"flags/admin.go:115","msg":"Starting admin HTTP server","http-addr":":14269"} {"level":"info","ts":1639750313.6564634,"caller":"flags/admin.go:96","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"} {"level":"info","ts":1639750313.6597028,"caller":"kafka/factory.go:69","msg":"Kafka factory","producer builder":{"Brokers":["pcdp-k01.bancogalicia.com.ar:9092","pcdp-k02.bancogalicia.com.ar:9092","pcdp-k03.bancogalicia.com.ar:9092","pcdp-k04.bancogalicia.com.ar:9092"],"RequiredAcks":1,"Compression":0,"CompressionLevel":0,"ProtocolVersion":"","BatchLinger":0,"BatchSize":0,"BatchMinMessages":0,"BatchMaxMessages":0,"Authentication":"kerberos","Kerberos":{"ServiceName":"kafka","Realm":"BGCMZ.BANCOGALICIA.COM.AR","UseKeyTab":true,"Username":"[email protected]","ConfigPath":"/etc/krb5.conf","KeyTabPath":"/etc/security/kafka.keytab"},"TLS":{"Enabled":false,"CAPath":"","CertPath":"","KeyPath":"","ServerName":"","ClientCAPath":"","SkipHostVerify":false},"PlainText":{"Username":"","Mechanism":"PLAIN"}},"topic":"jaeger-spans"} {"level":"fatal","ts":1639750314.6189425,"caller":"./main.go:80","msg":"Failed to init storage factory","error":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)","stacktrace":"main.main.func1\n\t./main.go:80\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/[email protected]/command.go:856\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/[email protected]/command.go:974\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/[email protected]/command.go:902\nmain.main\n\t./main.go:147\nruntime.main\n\truntime/proc.go:255"}
The cluster is working with other app. Any clue?
Thanks in advance.
Please stop disclosing/leaking internal server information
