Kafka Connect: ClassNotFoundException for plugin when bumping from 5.1.2 to 5.3.1 (latest)

[mmarvick-convoy]

We have a plugin for Kafka Connect that used to load correctly, but now can't be found and results in a java.lang.ClassNotFoundException. We haven't changed anything about our build, except to bump to a newer base image of confluentinc/cp-kafka-connect.

We're setting ENV CONNECT_PLUGIN_PATH="/usr/share/java" within our Dockerfile, and I've verified that the env variable is set properly on the image. I can also confirm that the .jar that we want to load the class from is successfully making it to /user/share/java/KafkaVaultProvider for both our old and new Docker images.

Our old images on confluentinc/cp-kafka-connect:5.1.2 would load the plugin just fine:

November 26th, 12:05:33.404 | [2019-11-26 20:05:33,404] INFO Registered loader: PluginClassLoader{pluginLocation=file:/usr/share/java/KafkaVaultProvider/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
November 26th, 12:05:33.404 | [2019-11-26 20:05:33,404] INFO Added plugin 'com.convoyinc.KafkaVaultProvider.KafkaVaultProvider' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
November 26th, 12:05:33.385 | [2019-11-26 20:05:33,385] INFO Loading plugin from: /usr/share/java/KafkaVaultProvider (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
November 26th, 12:05:25.290 | CONNECT_CONFIG_PROVIDERS_VAULT_CLASS=com.convoyinc.KafkaVaultProvider.KafkaVaultProvider

Our new images on confluentinc/cp-kafka-connect:5.3.1 do not:

December 5th, 16:39:58.569 | org.apache.kafka.common.config.ConfigException: Invalid value java.lang.ClassNotFoundException: com.convoyinc.KafkaVaultProvider.KafkaVaultProvider for configuration Invalid config:com.convoyinc.KafkaVaultProvider.KafkaVaultProvider ClassNotFoundException exception occurred
December 5th, 16:39:58.568 | [main] ERROR org.apache.kafka.clients.admin.AdminClientConfig - ClassNotFoundException exception occurred: com.convoyinc.KafkaVaultProvider.KafkaVaultProvider
December 5th, 16:39:53.221 | CONNECT_CONFIG_PROVIDERS_VAULT_CLASS=com.convoyinc.KafkaVaultProvider.KafkaVaultProvider

[Samlinxia]

I have the same issue. Any version above 5.3.0 will have this ClassNotFoundException on custom ConfigProvider.

[VimalKumarS]

Refer https://github.com/confluentinc/cp-docker-images/issues/828