hudi
hudi copied to clipboard
[HUDI-9154] Allow bootstrap to complete without updating col stats
Change Logs
Since #12529 , BaseCommitActionExecutor would update col stats by default. But bootstrap operation doesn't support col stats and will fail the bootstraping
- https://github.com/apache/hudi/blob/63362ad7b69d0bea50caceb1e744c53454a1cc23/hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/bootstrap/SparkBootstrapCommitActionExecutor.java#L225
This PR is to allow bootstrap to complete without updating col stats.
Caused by: org.apache.hudi.exception.HoodieNotSupportedException: col stats is not supported with bootstrap operation
at org.apache.hudi.table.action.bootstrap.SparkBootstrapCommitActionExecutor.updateColumnsToIndexForColumnStats(SparkBootstrapCommitActionExecutor.java:225)
at org.apache.hudi.table.action.commit.BaseCommitActionExecutor.lambda$commit$b950a45b$1(BaseCommitActionExecutor.java:239)
at org.apache.hudi.client.HoodieColumnStatsIndexUtils.updateColsToIndex(HoodieColumnStatsIndexUtils.java:73)
... 63 more
Impact
None
Risk level (write none, low medium or high below)
None
Documentation Update
None
- The config description must be updated if new configs are added or the default value of the configs are changed
- Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the instruction to make changes to the website.
Contributor's checklist
- [ ] Read through contributor's guide
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
#12977 seems to be able to fix this issue, I have not tested it yet. I'll test it later
CI report:
- c2ad1e287ff2a544d8ea09ab4ff10fe0cbd4033b UNKNOWN
- d29bc45097ba835b54acda1fb9e4af4a8c2dc856 Azure: FAILURE
Bot commands
@hudi-bot supports the following commands:@hudi-bot run azurere-run the last Azure build
This issue exists on the released Hudi 1.0.2 as well and will be needed