hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-7752] Abstract serializeRecords for log writing

Open yihua opened this issue 1 year ago • 2 comments

Change Logs

This PR adds a new API serializeRecordsToLogBlock to the FileFormatUtils class (renamed from BaseFileUtils), to abstract the serializeRecords logic in HoodieParquetDataBlock and HoodieHFileDataBlock.

Impact

Moves Hadoop-dependent logic of serializing Hudi records to log block content to the hudi-hadoop-common module.

Risk level

none

Documentation Update

none

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

yihua avatar May 14 '24 02:05 yihua

also not a fan of the org.apache.hudi.io.compress. package name. But probably too late to change now

Since the compression logic is also under the scope of IO, so we put the package name like this.

yihua avatar May 14 '24 03:05 yihua

CI report:

  • 4e3ae7175331848214a42f69ebadda04c5e9039e Azure: SUCCESS
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar May 14 '24 18:05 hudi-bot