[OPPRO-189] [WIP]Rebase Arrow to upstream Arrow-Change decompress code to adapt upstream Arrow

[jinchengchenghh]

oap-project Arrow change some field to private, and remove codec from VectorLoader, ignore the initial design, to fix that, change the decompress procedure to Java decompress

[github-actions[bot]]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[Gluten-${ISSUES_ID}] ${detailed message}

See also:

• Other pull requests

[jinchengchenghh]

This PR is ready. Can you help review? @JkSelf @zhztheplayer

[JkSelf]

@jinchengchenghh There may have some compile issue in gazelle backend. It may related with the changes in PR#372

[ERROR] /home/sparkuser/jk/projects/gluten/backends-gazelle/src/main/scala/io/glutenproject/backendsapi/velox/GazelleSparkPlanExecApi.scala:21: error: value SimpleColumnarRule is not a member of object org.apache.spark.sql.VeloxColumnarRules
[ERROR] import org.apache.spark.sql.VeloxColumnarRules.{DummyRule, SimpleColumnarRule}
[ERROR]        ^
[ERROR] /home/sparkuser/jk/projects/gluten/backends-gazelle/src/main/scala/io/glutenproject/backendsapi/velox/GazelleSparkPlanExecApi.scala:29: error: not found: value SimpleColumnarRule
[ERROR]     => SimpleColumnarRule(DummyRule, ArrowWritePostRule(spark))
[ERROR]        ^
[ERROR] /home/sparkuser/jk/projects/gluten/backends-gazelle/src/main/scala/io/glutenproject/backendsapi/velox/GazelleSparkPlanExecApi.scala:30: error: value genExtendedColumnarRules is not a member of io.glutenproject.backendsapi.velox.VeloxSparkPlanExecApi
[ERROR]     super.genExtendedColumnarRules():+ arrowRule
[ERROR]           ^
[ERROR] three errors found

[jinchengchenghh]

@JkSelf thanks, fixed, please try again

[zhztheplayer]

SimpleColumnarRule

The class was already removed from Gluten. It can be a git rebase issue