terraform-provider-kafka
terraform-provider-kafka copied to clipboard
Mongey
Feature request: Proxying
It happens, somewhat frequently, that I need to manage a Kafka cluster, one which cannot be connected directly to. I'd prefer to manage the cluster using Terraform, but since no Kafka providers support any form of proxying, this isn't really possible.
Looking at the underlying library used here, I see that Sarama's Config object supports proxying by way of the proxy.Dialer.
It would be awesome to be able to override Sarama's Config object, in order to provide proxy support.
Is this something that would be accepted into the codebase of this provider? If so, I'd very much like to donate time to build it!
I think that's something reasonable to support... I don't have the time to add this myself, but a PR is welcome 😄
It seems like it wouldn't be too much trouble implementing it. The only issue I'm having with it is that I can't get the upstream acceptance tests to run successfully under MacOS using Docker Desktop for Mac... Tracking it down, it seems to be an TLS authentication issue.
Am I right to assume the tests have never been run on MacOS? Might have time to try running the tests from a Linux host during the weekend... Knowing the tests will run successfully prior to writing any code is kinda nice...
One way to do this is to use a kafka-aware proxy and point your bootstrap server setting at it. Deploy the proxy somewhere that has connectivity to the cluster.
https://github.com/grepplabs/kafka-proxy
It seems like it wouldn't be too much trouble implementing it. The only issue I'm having with it is that I can't get the upstream acceptance tests to run successfully under MacOS using Docker Desktop for Mac... Tracking it down, it seems to be an TLS authentication issue.
@MadsRC tbh the TLS setup here is a bit of a mess, but whatever setup I have locally seems pass -- I'm using macOS + docker too. CI passes too. Is there an error you're getting ?