Kafka Transport throws Class not found error

[olqs]

Hi, using the module with configured kafka transport in combination with a wildfly or jboss server throws following error on startup: LogManager error of type OPEN_FAILURE: Could not send GELF message: Invalid value org.apache.kafka.common.serialization.ByteArraySerializer for configuration key.serializer: Class org.apache.kafka.common.serialization.ByteArraySerializer could not be found.

This error occurs with logstash-gelf in different versions, as well as with different kafka-client modules.

The standalone.xml logging configuration

         <custom-handler name="GelfLogger" class="biz.paluch.logging.gelf.wildfly.WildFlyGelfLogHandler" module="biz.paluch.logging">
               <properties>
                   <property name="version" value="1.0"/>
                   <property name="facility" value="logstash-gelf"/>
                   <property name="extractStackTrace" value="true"/>
                   <property name="filterStackTrace" value="true"/>
                   <property name="includeLogMessageParameters" value="true"/>
                   <property name="mdcProfiling" value="true"/>
                   <property name="timestampPattern" value="yyyy-MM-dd HH:mm:ss,SSS"/>
                   <property name="maximumMessageSize" value="8192"/>
                   <property name="includeFullMdc" value="true"/>
                   <property name="host" value="kafka://logginghost1:9092#mytopic"/>
               </properties>
           </custom-handler>

We added the kafka-clients module to the same folder as the logstash-gelf module and added it as resource in the module.xml. Is there anything more needed?

[mp911de]

Please make sure to include all Kafka jars in the WildFly module descriptor (or reference existing modules) so that the WildFly Module ClassLoader sees all required classes.

[olqs]

I have included the kafka-clients-...jar in the module.xml and the class org.apache.kafka.common.serialization.ByteArraySerializer is in this jar file. Is another jar needed in the module.xml?

[mp911de]

I'm not familiar with Kafka nor with the recent WildFly setup, so this is an issue to be sorted out within WildFly.