[KYUUBI #6348] Upgrade iceberg from 1.5.0 to 1.5.1

[mrpengbei]

:mag: Description

Issue References 🔗

This pull request fixes #6348

Describe Your Solution 🔧

In the project POM file, I have updated the Apache Iceberg version from 1.5.0 to 1.5.1

Types of changes :bookmark:

• [ ] Bugfix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request :coffin:

Behavior With This Pull Request :tada:

Related Unit Tests

---

Checklist 📝

• [x] This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

[wForget]

It looks like there may be scala version conflict.

*** RUN ABORTED ***
  java.lang.NoClassDefFoundError: scala/collection/SeqOps
  at org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
  at org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:174)
  at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
  at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
  at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
  at scala.collection.TraversableLike.map(TraversableLike.scala:286)
  at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
  at scala.collection.AbstractTraversable.map(Traversable.scala:108)
  at org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:174)
  ...
  Cause: java.lang.ClassNotFoundException: scala.collection.SeqOps
  at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
  at org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
  at org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:174)
  at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
  at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
  at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)

[wForget]

@mrpengbei There seems to be a problem with the iceberg-spark-runtime-3.3_2.12-1.5.jar. We can first specify iceberg.version=1.5.0 in the spark-3.3 profile. and cc @cxzl25 @pan3793

[wForget]

A similar error occurs in iceberg-spark-runtime-3.5 with scala 2.13:

*** RUN ABORTED ***
  java.lang.NoClassDefFoundError: scala/Serializable
  at java.lang.ClassLoader.defineClass1(Native Method)
  at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
  at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
  at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
  at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
  ...
  Cause: java.lang.ClassNotFoundException: scala.Serializable
  at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
  at java.lang.ClassLoader.defineClass1(Native Method)
  at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
  at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
  at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
  at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
  ...

[pan3793]

@mrpengbei Iceberg 1.5.2 is available, please try 1.5.2

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 58.48%. Comparing base (5cbbdc3) to head (9303898). Report is 16 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #6349      +/-   ##
============================================
+ Coverage     58.45%   58.48%   +0.03%     
  Complexity       24       24              
============================================
  Files           653      653              
  Lines         39865    39959      +94     
  Branches       5481     5487       +6     
============================================
+ Hits          23303    23371      +68     
- Misses        14073    14086      +13     
- Partials       2489     2502      +13     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[pan3793]

Thanks, merged to master/1.9