kafka-connect-protobuf-converter icon indicating copy to clipboard operation
kafka-connect-protobuf-converter copied to clipboard

Unclear how to set protoClassName

Open bkinzle opened this issue 2 years ago • 1 comments

I think my protoClassName should be mdg.eng.proto.Kafka$StatsValue based on the note:

Note: Nested classes must be specified using the $ notation, for example com.blueapron.connect.protobuf.NestedTestProtoOuterClass$NestedTestProto

If I cat my jar the first few lines look like this:

$ cat 45a54b58.jar
PK
!&o�22META-INF/MANIFEST.MFManifest-Version: 1.0
Created-By: 1.6.0 (protoc)

PK
!G~�<
     <
      mdg/eng/proto/Kafka.java// Generated by the protocol buffer compiler.  DO NOT EDIT!
// source: kafka.proto

package mdg.eng.proto;

public final class Kafka {
  private Kafka() {}
  public static void registerAllExtensions(
      com.google.protobuf.ExtensionRegistryLite registry) {
  }

My proto jar is located /opt/kafka/plugins/mdg-eng-proto/45a54b58/45a54b58.jar And the jars for this project are located at /opt/kafka/plugins/blueapron-protobuf-converter/4da72fe6

I'm trying to use the Strimzi operator to build my Kafka Connect container image which limits my flexibility as to where I can put these things.

I'm getting errors like this:

org.apache.kafka.connect.errors.ConnectException: Proto class mdg.eng.proto.Kafka$StatsValue not found in the classpath
    at com.blueapron.connect.protobuf.ProtobufConverter.configure(ProtobufConverter.java:48)
    at org.apache.kafka.connect.runtime.isolation.Plugins.newConverter(Plugins.java:297)
    at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:642)
    at org.apache.kafka.connect.runtime.Worker.startSinkTask(Worker.java:544)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:1705)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$32(DistributedHerder.java:1755)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

Any idea what I've got wrong here?

bkinzle avatar Dec 01 '22 08:12 bkinzle

  1. Check classes in jar file jar tvf your.jar
  2. Cast protoClassName. package.MAINclass$NESTEDclass

MilkyWay-core avatar Jun 09 '23 09:06 MilkyWay-core