hudi
hudi copied to clipboard
[MINOR] if sync mode is glue, fix the sync tool class
Change Logs
While writing the Hudi table in spark, I want to sync the table to Glue.
However, the sync mode: "hoodie.datasource.hive_sync.mode": "glue"
is actually not applicable in spark.
It instead works using this setting: "hoodie.meta.sync.client.tool.class": "org.apache.hudi.aws.sync.AwsGlueCatalogSyncTool"
.
I believe that the former config offers a better developer experience and is rather cheap to implement. Raising the PR for same as well.
Impact
Some spark code might spurious hoodie option of "hoodie.datasource.hive_sync.mode": "glue"
but is actually writing to some other hive metastore. The behavior might start breaking.
Risk level (write none, low medium or high below)
na
Documentation Update
"none"
Contributor's checklist
- [x] Read through contributor's guide
- [x] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed