hudi
hudi copied to clipboard
[WIP][HUDI-4449] Support DataSourceV2 Read for Spark3.2
Change Logs
NA
Impact
NA
Risk level (write none, low medium or high below)
medium
Contributor's checklist
- [ ] Read through contributor's guide
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
does this PR only aimed to support spark 3.2.0 not spark 3.2.1/3.2.2 or 3.3.0? or with following-up tasks to support other versions?
@leesf thanks for your time to review this, I found that there are serveral problems with this pr, since upstream has revert v2 datasource, https://github.com/apache/hudi/blob/3adb5715310b372c2e0330102169c40d1c7448ba/hudi-spark-datasource/hudi-spark3.2.x/src/main/scala/org/apache/hudi/Spark3DefaultSource.scala#L28, I'll update soon
@alexeykudinkin FYI
@leesf @alexeykudinkin @XuQianJin-Stars Thanks for your time. I've been busy with other things these days, sorry for the late reply.