kyuubi
kyuubi copied to clipboard
[KYUUBI #6633][Bug] Running with paimon failed with class not found exception
:mag: Description
Issue References ๐
This pull request fixes # Running spark + auth + paimon extension throw below exception
java.lang.NoClassDefFoundError: [Lorg/apache/spark/sql/connector/catalog/Column;
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at org.apache.kyuubi.plugin.spark.authz.util.AuthZUtils$.invoke(AuthZUtils.scala:63)
at org.apache.kyuubi.plugin.spark.authz.util.AuthZUtils$.invokeAs(AuthZUtils.scala:77)
at org.apache.kyuubi.plugin.spark.authz.serde.TableExtractor$.getOwner(tableExtractors.scala:52)
at org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor.$anonfun$apply$8(tableExtractors.scala:142)
at scala.Option.flatMap(Option.scala:271)
at org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor.apply(tableExtractors.scala:140)
at org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor.apply(tableExtractors.scala:129)
at org.apache.kyuubi.plugin.spark.authz.serde.ScanDesc.extract(Descriptor.scala:307)
at org.apache.kyuubi.plugin.spark.authz.serde.ScanSpec.$anonfun$tables$2(CommandSpec.scala:102)
at scala.collection.immutable.List.flatMap(List.scala:366)
at org.apache.kyuubi.plugin.spark.authz.serde.ScanSpec.$anonfun$tables$1(CommandSpec.scala:100)
at org.apache.kyuubi.plugin.spark.authz.ram.RamPrivilegesBuilder$.buildQuery(RamPrivilegesBuilder.scala:164)
Caused by paomin low version extension jar SparkTable will carry high version spark class
Describe Your Solution ๐ง
Make the extract process more robust
Types of changes :bookmark:
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
Test Plan ๐งช
Behavior Without This Pull Request :coffin:
Behavior With This Pull Request :tada:
Related Unit Tests
Checklist ๐
- [ ] This patch was not authored or co-authored using Generative Tooling
Be nice. Be informative.