hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-8313] Adding a new sync tool to integrate with lakeview

Open pkgajulapalli opened this issue 1 year ago • 1 comments
trafficstars

Change Logs

Created a new sync-tool to sync hoodie tables with Onehouse's lakeview. This also includes another set of sync-tools to sync hoodie tables in both lakeview & hive/glue. These composite sync tools will be helpful in standard Spark jobs where hudi only supports only one SyncTool.

Impact

Users will be able to sync their hoodie tables with Lakeview as well. This will help users in capturing table level metrics such as table size, number of partitions, file groups etc.

Risk level (write none, low medium or high below)

low

Documentation Update

Documentation update is required for detailed instructions related using LakeViewSyncTool & using it along with Hive/Glue SyncTools.

https://issues.apache.org/jira/browse/HUDI-8313

Contributor's checklist

  • [x] Read through contributor's guide
  • [x] Change Logs and Impact were stated clearly
  • [x] Adequate tests were added if applicable
  • [ ] CI passed

pkgajulapalli avatar Oct 06 '24 03:10 pkgajulapalli

CI report:

  • a63fcd812d67b1800933763444b20b9bb98950f6 Azure: FAILURE
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Oct 13 '24 07:10 hudi-bot