Support OAuthBearer Hooks

[psycotica0-shopify]

Once https://github.com/edenhill/librdkafka/issues/2123 ships, librdkafka will have the ability to support the oauthbearer SASL model.

It would be lovely if this was exposed to kafkacat, but the mechanism is a bit tricky, perhaps. Unlike other auth systems, oauth tokens are short-lived and expire (typically).

So we could expose an option that provides a token, but the connections might die after an hour (for example).

I don't know what the right path is from here, since every other library provides hooks that this logic hooks into, whereas this is a binary. I feel like we don't want to link in a .so file or something 😄

Other ideas:

1. Point to a filename, and some external process makes sure that the file always contains a valid token
2. Point to an executable, and kafkacat will run it when it needs a token. The executable prints a token to stdout?

I think the file version sounds the cleanest, but I'd be interested in hearing other ideas.

[edenhill]

Interesting ideas.

I do believe this requires at least one or two actual use-cases as a proof-of-concept that it will work in practice.

[psycotica0-shopify]

My use-case is that I use kafkacat nearly every day at my work, and we're changing our Kafka clusters to use oauth-bearer to authenticate with the cluster. *edit* I should also mention that our tokens expire after an hour

So I figured I'd start the conversation now on what the right way to handle it was before I start a PR in the next few months 😄 (at least that's the plan)

[edenhill]

I think the easiest approach here is to pass a path to a file which contains the token, and setting that token explicitly with set_token() upon kafkacat startup with a max lifetime, so that no refreshes are needed.

ping @rondagostino

[edenhill]

I've added experimental support to specify the token by file, please try it out on the oauth branch.

$ kafkacat -b mybrokers -L -X security.protocol=SASL_PLAINTEXT -X oauthbearer.token.file=YourFile -X oauthbearer.token.lifetime=12345 -X oauthbearer.token.principal=admin

[mdharper]

Is there an example of the final version of the oauthbearer integration? oauthbearer.token.file does not appear to be a valid property in the current version.

[igorcalabria]

Hi, is there any plans to merge the oauth branch? Reading the token file is a legit usecase for us since that's how kubernetes mounts the service account tokens. The branch only required a small patch, but seems to be working just fine after that

diff --git a/kafkacat.c b/kafkacat.c
index fcf82a7..b02b8bd 100644
--- a/kafkacat.c
+++ b/kafkacat.c
@@ -30,6 +30,7 @@
 #include <unistd.h>
 #include <syslog.h>
 #include <sys/time.h>
+#include <time.h>
 #include <sys/mman.h>
 #else
 #pragma comment(lib, "ws2_32.lib")

I just have a couple questions regarding the auth params:

• Isn't oauthbearer.token.principal redundant since the principal will be the sub field in the token?
• Is there a reason why token lifetime wasn't extracted from the exp field in the token file?

[ehm-93]

Hey @edenhill if you're still searching for a concrete use case before prioritizing this, AWS recently enabled IAM integration with MSK via SASL/OAUTHBEARER which would be very convenient to plug kcat into.