kyuubi icon indicating copy to clipboard operation
kyuubi copied to clipboard

[KYUUBI #6633][Bug] Running with paimon failed with class not found exception

Open AngersZhuuuu opened this issue 6 months ago โ€ข 5 comments

:mag: Description

Issue References ๐Ÿ”—

This pull request fixes # Running spark + auth + paimon extension throw below exception

java.lang.NoClassDefFoundError: [Lorg/apache/spark/sql/connector/catalog/Column;
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
	at java.lang.Class.getMethod0(Class.java:3018)
	at java.lang.Class.getMethod(Class.java:1784)
	at org.apache.kyuubi.plugin.spark.authz.util.AuthZUtils$.invoke(AuthZUtils.scala:63)
	at org.apache.kyuubi.plugin.spark.authz.util.AuthZUtils$.invokeAs(AuthZUtils.scala:77)
	at org.apache.kyuubi.plugin.spark.authz.serde.TableExtractor$.getOwner(tableExtractors.scala:52)
	at org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor.$anonfun$apply$8(tableExtractors.scala:142)
	at scala.Option.flatMap(Option.scala:271)
	at org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor.apply(tableExtractors.scala:140)
	at org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor.apply(tableExtractors.scala:129)
	at org.apache.kyuubi.plugin.spark.authz.serde.ScanDesc.extract(Descriptor.scala:307)
	at org.apache.kyuubi.plugin.spark.authz.serde.ScanSpec.$anonfun$tables$2(CommandSpec.scala:102)
	at scala.collection.immutable.List.flatMap(List.scala:366)
	at org.apache.kyuubi.plugin.spark.authz.serde.ScanSpec.$anonfun$tables$1(CommandSpec.scala:100)
	at org.apache.kyuubi.plugin.spark.authz.ram.RamPrivilegesBuilder$.buildQuery(RamPrivilegesBuilder.scala:164)

Caused by paomin low version extension jar SparkTable will carry high version spark class

Describe Your Solution ๐Ÿ”ง

Make the extract process more robust

Types of changes :bookmark:

  • [x] Bugfix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Test Plan ๐Ÿงช

Behavior Without This Pull Request :coffin:

Behavior With This Pull Request :tada:

Related Unit Tests


Checklist ๐Ÿ“

Be nice. Be informative.

AngersZhuuuu avatar Aug 21 '24 02:08 AngersZhuuuu