hudi icon indicating copy to clipboard operation
hudi copied to clipboard

Published 20 hours ago • verified publisher icon apache

[HUDI-6497] WIP HoodieStorage abstraction

Open yihua opened this issue 1 year ago • 1 comments
trafficstars

Change Logs

This PR introduces HoodieStorage abstraction and tries to remove the usage of Hadoop File System classes (org.apache.hadoop.fs.[FileSystem, Path, FileStatus], etc.) in core Hudi APIs, so it's much easier to plugin different file system implementation.

This is WIP.

Impact

Improves storage abstraction and APIs in Hudi.

Risk level

medium

Documentation Update

New abstraction and API changes for developers.

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

yihua avatar Dec 19 '23 02:12 yihua

CI report:

  • 0a958d6408a7d0107ae2dcfc2aae676fd1a6977d UNKNOWN
  • 6632d6e715eec0e54ae047f1d89c8f979ac8639d UNKNOWN
  • fd05a7d87c676275e5f5e329e0207cc97ec9adfb UNKNOWN
  • 74b8a6658f324313bec3525aae40a3203a8c6bc1 Azure: FAILURE
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Jan 25 '24 04:01 hudi-bot

This PR is replaced by #10591 as the last main piece of the storage abstraction.

yihua avatar Mar 06 '24 07:03 yihua