StackOverflow Exception while using my own protoClassName having field type google.protobuf.Struct

[akanungoz]

Hi, I'm using protobuf-converter for S3 sink connector for ingesting protobuf from kafka to S3. my sink config looks like - Map.entry("connector.class", "io.confluent.connect.s3.S3SinkConnector"), Map.entry("topics", "topicName"), Map.entry("tasks.max", 1), Map.entry("flush.size", 100), Map.entry("rotate.schedule.interval.ms", 10000), Map.entry("s3.region", s3Region), Map.entry("s3.bucket.name", s3Bucket), Map.entry("s3.ssea.name", s3Ssea), Map.entry("s3.part.size", 26214400), Map.entry("storage.class", "io.confluent.connect.s3.storage.S3Storage"), Map.entry("key.converter", "com.blueapron.connect.protobuf.ProtobufConverter"), Map.entry("value.converter", "com.blueapron.connect.protobuf.ProtobufConverter"), Map.entry("value.converter.protoClassName", "com.protobuf.hawaii.Event"), Map.entry("format.class", "io.confluent.connect.s3.format.json.JsonFormat"), Map.entry("schemas.enable", "true"), Map.entry("schema.compatibility", "NONE"), Map.entry("partitioner.class", "io.confluent.connect.storage.partitioner.TimeBasedPartitioner"), Map.entry("timestamp.extractor", "Wallclock"), Map.entry("partition.duration.ms", 3600000), Map.entry("topics.dir", String.format("%s/connect_topics", s3Root)), Map.entry("path.format", "'year'=YYYY/'month'=MM/'day'=dd/'hour'=HH"), Map.entry("locale", "en-US"), Map.entry("timezone", "UTC"));

I'm getting this exception when I try to run this - StackOverflowError 1 at internalGetFieldAccessorTable (com.google.protobuf.DescriptorProtos$FieldOptions.java:27472) 2 at getAllFieldsMutable (com.google.protobuf.GeneratedMessageV3.java:155) 3 at access$800 (com.google.protobuf.GeneratedMessageV3.java:79) 4 at getAllFields (com.google.protobuf.GeneratedMessageV3$ExtendableMessage.java:1232) 5 at getConnectFieldName (com.blueapron.connect.protobuf.ProtobufData.java:72) 6 at toConnectSchema (com.blueapron.connect.protobuf.ProtobufData.java:194) 7 at toConnectSchema (com.blueapron.connect.protobuf.ProtobufData.java:194) 8 at toConnectSchema (com.blueapron.connect.protobuf.ProtobufData.java:194) 9 at toConnectSchema (com.blueapron.connect.protobuf.ProtobufData.java:194) ..........

Edit - I think the issue is when you have google.protobuf.Struct type in .proto/schema file. It throws the above exception. Could you look around this and help to support google.protobuf.Struct types?

[akanungoz]

@sschatts could you please update on this one?

[sschatts]

@akanungoz this is on our backlog. We will update this ticket.

[gudladona]

Any update on this?