kafka-connect-transform-common icon indicating copy to clipboard operation
kafka-connect-transform-common copied to clipboard

fix: implement processMap function to MAP structured data

Open peacecwz opened this issue 2 years ago • 7 comments

Hi @jcustenborder,

Maybe you saw If you are using HeaderToField as transforms and your data struct is Map, the plugin is throwing error like "MAP is unsupported ..." I implemented processMap function and I tried to use it and It works well. Maybe you want to merge it as contribution

peacecwz avatar May 10 '23 09:05 peacecwz

@peacecwz Happy to merge this! Would you mind adding some unit tests?

jcustenborder avatar May 10 '23 14:05 jcustenborder

@jcustenborder Actually no but I can add some unit tests for the function. I'll update the PR quickly

peacecwz avatar May 10 '23 14:05 peacecwz

@jcustenborder I added test for processMap Is that okay for that?

peacecwz avatar May 10 '23 14:05 peacecwz

@jcustenborder btw If this PR will be merged can you also make a release it? Because I'm using my private artifact. I couldn't run Jenkins pipeline. It uploaded the artifact manually to S3 and deploy it as well but I would like to deploy with confluent-kafka CLI as officially

peacecwz avatar May 10 '23 15:05 peacecwz

@jcustenborder Hey can you check the PR?

peacecwz avatar May 25 '23 12:05 peacecwz

@peacecwz

Could you fix same issue on the ChangeCase SMT..?

(If not.. I will try to implement processMap method on the ChangeCase, by referring to your commit.)

Caused by: java.lang.UnsupportedOperationException: MAP is not a supported type.
	at com.github.jcustenborder.kafka.connect.transform.common.BaseTransformation.processMap(BaseTransformation.java:39)
	at com.github.jcustenborder.kafka.connect.transform.common.BaseTransformation.process(BaseTransformation.java:120)
	at com.github.jcustenborder.kafka.connect.transform.common.ChangeCase$Value.apply(ChangeCase.java:128)
	at org.apache.kafka.connect.runtime.TransformationChain.lambda$apply$0(TransformationChain.java:50)
	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:180)
	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:214)
	... 15 more

okayhooni avatar Oct 20 '23 11:10 okayhooni

@peacecwz see https://github.com/peacecwz/kafka-connect-transform-common/pull/1

greyfairer avatar Apr 10 '24 14:04 greyfairer