cloudflow
cloudflow copied to clipboard
lightbend
Update kafka version used for integration testing and in docs to 2.8.1
What changes were proposed in this pull request?
Update the strimzi-code samples in documentation and the test-infrastructure to explicitly use Kafka 2.8.1. The build-tool and integ-test code now also contains hints what to do and where to look at, when trying to achieve compatibility with Arm64 hosts (in the cloud, Apple M1/M2, Raspberry Pis, etc.).
Why are the changes needed?
The strimzi-sample does not mention an explicit Kafka version, so it's up the to the Cloudflow users to assume which version to go with. We have successfully used Cloudflow 2.2.2 in production for many months in combo with Kafka 2.8.1 already.
This is also the furthest we can go while still maintaining Scala 2.12, since alpakka-kafka 3.x does not support Scala 2.12 anymore. -> A decision is needed here!
Does this PR introduce any user-facing change?
Documentation was updated and suggests having Strimzi create a Kafka 2.8.1 broker instance.
How was this patch tested?
Unit and integration tests
@franciscolopezsancho Since you just merged some PRs and worked on Cloudflow. The current documentation is broken as it suggests installing the latest version of strimzi.
This does not work anymore, because latest strimzi (0.31.0) supports Kafka 3.2.1 but no older versions of Kafka. https://github.com/strimzi/strimzi-kafka-operator/blob/main/CHANGELOG.md
At the same time, Cloudflow can only support up to Kafka 2.8.1 due to its dependency on alpakka-kafka 2.x. - Upgrading to alpakka-kafka 3.x is not possible right now, because Cloudflow would need to drop Scala 2.12 support for that.
Actually, with Cloudflow 2.3.2 using "kafka-clients" % "3.2.1", I wonder if Cloudflow works with Kafka 3.x?
Does "akka-stream-kafka" % 2.1.1 support Kafka 3?
At least one commit author ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]) is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user
