Kafka-clients: writing automated tests

Run all tests:

./mvnw clean install -DskipIntegrationTests=false

Modules and approaches

1. streams-client module contains examples of unit-tests for kafka-streams topologies with kafka-streams-test-utils. Approach covers testing topologies (stateful & stateless processors) with different serdes including avro and confluent schema registry.
2. embedded-kafka-cluster module is example of kafka-embedded cluster in memory (1 Zookeeper, 1 Kafka broker, 1 Confluent schema registry). Embedded kafka cluster is used for integration test of kafka-client application.
3. consumer-producer-clients module contains examples of integration tests with embedded kafka cluster and kafka based applications with Producer/Consumer API
4. data-pipeline module contains examples of integration tests with embedded kafka cluster, wire-mock and kafka based applications with Streams API
5. e2e module contains IT test for data pipeline, using testcontainers

TODO:

• [ ] Makefile
• [ ] update vers java 8 -> 12 !NB: Reflection use -> only java8 currently
• [ ] update vers junit 4 -> 5

Important notes

• Confluent Platform and Apache Kafka Compatibility

References

• Apache Kafka. Developer guide. Testing
• Getting Your Feet Wet with Stream Processing – Part 2: Testing Your Streaming Application