AWS MSK and TLS trusted source

[fireside-social]

I am getting a "Connection Error: Unable to bootstrap" from my AWS MSK broker using the TLS 9094 port

This is likely because of the setting in AWS MSK for requiring TLS between clients and brokers the apache docs indicate we should use a truststore.jks for this

ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks bin/kafka-console-consumer --bootstrap-server kafka1:9093 --topic test --consumer.config client-ssl.properties --from-beginning

Has anyone had success with this? I'm not having any luck using the ssl options below https://faust.readthedocs.io/en/latest/userguide/settings.html#ssl-authentication configuration

[danigosa]

I have successfully done SASL_SSL with Azure EventsHub with similar TLS issues from certificates and it turned out to be that you have to manually add well known CA's (from Mozilla), not sure if that will solve AWS but it might worth a try:

sasl_username: str = "xxxx"
sasl_password: str = "xxxx"
ssl_cert_path: str = "cacert.pem"

faust.App(
 self.name,
 broker=broker_url,
 broker_credentials=faust.SASLCredentials(
   username=sasl_username,
   password=sasl_password,
   ssl_context=ssl.create_default_context(
       purpose=ssl.Purpose.SERVER_AUTH, cafile=ssl_cert_path
   ),
 ),
#[...]

Mozilla CA's download file: https://curl.haxx.se/docs/caextract.html

[rubenesp87]

This worked for me adding ssl context without cert

import ssl

context = ssl.SSLContext()
context.verify_mode=ssl.CERT_NONE
context.check_hostname = False

app = faust.App(..., broker_credentials=context)

[chase2981]

To all who come across this in an attempt to integrate faust with aws msk, you should know faust is no longer actively maintained. See https://github.com/robinhood/faust/issues/707 for more details.