hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[MINOR] Validate that fg reader is reading entire files for bootstrap and mor

Open jonvex opened this issue 1 year ago • 2 comments

Change Logs

check to make sure that the fg reader is reading from the start of base files for bootstrap and mor merging. For bootstrap, the skeleton file and base file will probably not line up. For mor, we will read all the log files for each split and that will be slow and probably cause data to be wrong.

Impact

Validate that fg reader is being used correctly

Risk level (write none, low medium or high below)

none

Documentation Update

N/A

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

jonvex avatar Aug 16 '24 19:08 jonvex

@hudi-bot run azure

yihua avatar Aug 16 '24 20:08 yihua

CI report:

  • 19466d84da356ea8da1053f1a4cf2513a3a66c77 UNKNOWN
  • d730787fa9fc002221dfc8e1ffb21ec4349fbcf0 Azure: SUCCESS
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Sep 04 '24 20:09 hudi-bot