hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-9154] Allow bootstrap to complete without updating col stats

Open CTTY opened this issue 8 months ago • 3 comments

Change Logs

Since #12529 , BaseCommitActionExecutor would update col stats by default. But bootstrap operation doesn't support col stats and will fail the bootstraping

  • https://github.com/apache/hudi/blob/63362ad7b69d0bea50caceb1e744c53454a1cc23/hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/bootstrap/SparkBootstrapCommitActionExecutor.java#L225

This PR is to allow bootstrap to complete without updating col stats.

Caused by: org.apache.hudi.exception.HoodieNotSupportedException: col stats is not supported with bootstrap operation
	at org.apache.hudi.table.action.bootstrap.SparkBootstrapCommitActionExecutor.updateColumnsToIndexForColumnStats(SparkBootstrapCommitActionExecutor.java:225)
	at org.apache.hudi.table.action.commit.BaseCommitActionExecutor.lambda$commit$b950a45b$1(BaseCommitActionExecutor.java:239)
	at org.apache.hudi.client.HoodieColumnStatsIndexUtils.updateColsToIndex(HoodieColumnStatsIndexUtils.java:73)
	... 63 more

Impact

None

Risk level (write none, low medium or high below)

None

Documentation Update

None

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the instruction to make changes to the website.

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

CTTY avatar Mar 10 '25 22:03 CTTY

#12977 seems to be able to fix this issue, I have not tested it yet. I'll test it later

CTTY avatar Mar 26 '25 04:03 CTTY

CI report:

  • c2ad1e287ff2a544d8ea09ab4ff10fe0cbd4033b UNKNOWN
  • d29bc45097ba835b54acda1fb9e4af4a8c2dc856 Azure: FAILURE
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Mar 26 '25 06:03 hudi-bot

This issue exists on the released Hudi 1.0.2 as well and will be needed

CTTY avatar May 20 '25 00:05 CTTY