apache
[Fix][Transform-v2] Fix custom udf throw exception
Purpose of this pull request
custom udf throw Exception. remove seatunnel-transform-v2.jar from connecots to lib solve the problem .
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/seatunnel/transform/sql/zeta/ZetaUDF
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56)
at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:97)
at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at java.lang.Iterable.forEach(Iterable.java:74)
at org.apache.seatunnel.transform.sql.zeta.ZetaSQLEngine.init(ZetaSQLEngine.java:88)
at org.apache.seatunnel.transform.sql.SQLTransform.open(SQLTransform.java:94)
at org.apache.seatunnel.transform.sql.SQLTransform.tryOpen(SQLTransform.java:103)
at org.apache.seatunnel.transform.sql.SQLTransform.transformTableSchema(SQLTransform.java:115)
at org.apache.seatunnel.transform.common.AbstractSeaTunnelTransform.transformCatalogTable(AbstractSeaTunnelTransform.java:69)
at org.apache.seatunnel.transform.common.AbstractSeaTunnelTransform.getProducedCatalogTable(AbstractSeaTunnelTransform.java:54)
at org.apache.seatunnel.transform.common.AbstractMultiCatalogTransform.lambda$new$3(AbstractMultiCatalogTransform.java:86)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.seatunnel.transform.common.AbstractMultiCatalogTransform.<init>(AbstractMultiCatalogTransform.java:88)
at org.apache.seatunnel.transform.common.AbstractMultiCatalogFlatMapTransform.<init>(AbstractMultiCatalogFlatMapTransform.java:33)
at org.apache.seatunnel.transform.sql.SQLMultiCatalogFlatMapTransform.<init>(SQLMultiCatalogFlatMapTransform.java:32)
at org.apache.seatunnel.transform.sql.SQLTransformFactory.lambda$createTransform$0(SQLTransformFactory.java:51)
at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareMultiTableTransform(FactoryUtil.java:445)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseTransform(MultipleTableJobConfigParser.java:459)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseTransforms(MultipleTableJobConfigParser.java:408)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:232)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:123)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:191)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:165)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.transform.sql.zeta.ZetaUDF
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 49 more
Does this PR introduce any user-facing change?
How was this patch tested?
e2e case : ExampleUdfIT
Check list
- [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
- [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
- [ ] If you are contributing the connector code, please check that the following files are updated:
- Update plugin-mapping.properties and add new connector information in it
- Update the pom file of seatunnel-dist
- Add ci label in label-scope-conf
- Add e2e testcase in seatunnel-e2e
- Update connector plugin_config
- [ ] Update the
release-note.
Thanks @hawk9821 point this issue, this is a bug, good catch.
It would be more appropriate to move interface org.apache.seatunnel.transform.sql.zeta.ZetaUDF to the api module
I tested with this fix, it works. But I don’t understand why these changes can be work. Can you give me some hint? Thanks.
BTW, when do this update, it can impact the user that has implment udf. it need update the udf lib, update the class full name, we need describe this in the document or release note (Breaking Change).
Thanks @hawk9821 point this issue, this is a bug, good catch.
It would be more appropriate to move interface org.apache.seatunnel.transform.sql.zeta.ZetaUDF to the api module
I tested with this fix, it works. But I don’t understand why these changes can be work. Can you give me some hint? Thanks.
BTW, when do this update, it can impact the user that has implment udf. it need update the udf lib, update the class full name, we need describe this in the document or release note (Breaking Change).
The compatibility issue was indeed not considered. It has been fixed in the way of moving transform-v2.jar to the lib .
Need to delete transform from plugin-mapping.properties?
https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties #L148
