[HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark

[jonvex]

trafficstars

Change Logs

• Create Spark Parquet reader inside of the reader context
• Eliminate parquet reader map
• Eliminate parallel implementation of setting up reader
• Schema.on.read for filegroup reader for spark
• Schema.on.read implemented for record buffers
• Make schema.on.read easier to port to future park versions
• Schema.on.write full type promotion support

Impact

Big step to making fg reader production ready

Risk level (write none, low medium or high below)

high need to do lots of testing

Documentation Update

N/A

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[jonvex]

Azure CI all passing @yihua

[hudi-bot]

CI report:

• 83064f1614c80f4ff518d05bf40563881ad12a5e Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build