[Feature request] Support for MSK & IAM integration

[bpuertolas]

Is your feature request related to a problem? Please describe. Recently, AWS released an integration between Amazon MSK (Amazon Managed Stream for Kafka) and IAM (Identity & Access Management). There is an example of this implementation but they're using a java client and a library named aws-msk-iam-auth in order to match IAM security.

Describe the solution you'd like Would like to fully support this new integration.

Additional context Currently, the only security measures that can be added to a Amazon MSK cluster is by protecting it with security groups or TLS. These options are full deny or full allow, there's no way to define a set of access per topic. This new integration with IAM allow this.

[gunturaf]

There's implementation reference for AWS_MSK_IAM at franz-go in this example https://github.com/twmb/franz-go/tree/master/examples/sasl/aws_msk_iam , I think it's a great addition to Sarama because downstream projects such as otel-contrib and Benthos uses Sarama as their underlying Kafka client, if Sarama supports this then the other projects will benefit too.

Can we sort of copy-pasta your AWS_MSK_IAM code in franz-go to achieve this in Sarama, @twmb ?

[cmaher]

kafka-go now has another reference implementation, which uses the aws-sdk to perform the signing, if the maintainers prefer that.

[twmb]

opentelemetry managed to work around this without any changes to sarama. https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/kafkaexporter/authentication.go#L127

[MovieStoreGuy]

This needs to be added to the main project, the otel contribution written by me is currently flawed since sarama does validation checks on the auth mechanism used.

This should be a simple as taking the current version in otel and making it part of the main library.

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[bpalermo]

@MovieStoreGuy any progress on this?

[dtjm]

AWS has released a solution here https://github.com/aws/aws-msk-iam-sasl-signer-go