Kafka - Incompatibility with configuration injection with environment variables

[mederel]

This issue was discovered while using the smallrye-reactive-messaging-kafka Quarkus extension with their QuarkusIntegrationTest annotation test with some configuration overrides. But it besically boils down to an issue of smallrye-reactive-messaging-provider to extract the different channel names from the smallrye-config root object.

Imagine a words-in channel is defined with @Incoming. We then want for an override of one of its properties with an environment variable. Example: MP_MESSAGING_INCOMING_WORDS_IN_RETRIES=1. That breaks the initialization of the provider package, because it interprets the environment variable as being a in.retries property for the words channel.

I have prepared a reproducer (using Quarkus): bug-smallrye-kafka-integration-config-override-2.zip

Modus operandi:

• get the zip and unzip
• run mvn clean install inside
• run MP_MESSAGING_INCOMING_WORDS_IN_RETRIES=1 java -jar target/quarkus-app/quarkus-run.jar and see the stacktrace:

java.lang.IllegalArgumentException: SRMSG00071: Invalid channel configuration -  the `connector` attribute must be set for channel `words`
        at io.smallrye.reactive.messaging.providers.impl.ConnectorConfig.lambda$new$0(ConnectorConfig.java:50)
        at java.base/java.util.Optional.orElseThrow(Optional.java:403)
        at io.smallrye.reactive.messaging.providers.impl.ConnectorConfig.lambda$new$1(ConnectorConfig.java:50)
        at java.base/java.util.Optional.orElseGet(Optional.java:364)
        at io.smallrye.reactive.messaging.providers.impl.ConnectorConfig.<init>(ConnectorConfig.java:49)
        at io.smallrye.reactive.messaging.providers.impl.ConfiguredChannelFactory.lambda$extractConfigurationFor$0(ConfiguredChannelFactory.java:86)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)
        at io.smallrye.reactive.messaging.providers.impl.ConfiguredChannelFactory.extractConfigurationFor(ConfiguredChannelFactory.java:75)

[cescoffier]

It's a known issue. I had an idea at some point to improve the detection in Quarkus (as we can read the annotations at build time and pass "hints" to reactive messaging). Unfortunately, I never had the time to implement it.

[cescoffier]

Another idea would be to check for well-known attributes used for all connectors (merge, broadcast, retries, enabled...)

[mederel]

Hi @cescoffier, I tried the idea of checking for well-known attributes, adding the connector one to the list, but then of course it changes the behaviour as not raising an exception for missing .connector configuration...

I fill like this is not the way to go. The impacts are almost impossible to guess and my confidence in the fix working without impacting the current behaviour is close to zero. I imagine that is why there is no fix today.

I had a question about the idea of reading annotations: why is that something we cannot apply to smallrye reactive messaging directly?

I push the attempt at a fix here.

[mederel]

I proposed a fix in Quarkus.