hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[SUPPORT]create hive table on top of existing hudi table

Open ayan2k21 opened this issue 1 year ago • 2 comments

I have a spark job which runs weekly and writes output to hudi table.Hive sync is not enabled currently. This job has been running for last 4 months. Now I want to create a hive table on top of the existing hudi table.

If I add hive sync properties in the hudi write in my job now is it going to create hive table and show existing data? If not what could be the possible solution.

Thanks in advance.

Environment AWS EMR 6.5 HUDI 0.13 SPARK 3.3.4

ayan2k21 avatar Aug 15 '24 18:08 ayan2k21

Just enable the hive sync would work, but because you enabled the hive sync all of a sudden, there might be too much metadata to sync which is a pressure for Hive(would be slow).

danny0405 avatar Aug 16 '24 00:08 danny0405

@ayan2k21 Let us know if you were able to enable hive sync and get the desired output. Let us know in case of any issues. Thanks.

ad1happy2go avatar Aug 22 '24 04:08 ad1happy2go

It worked. Job took little loner to run but all data is now visible in hive table.Thanks

ayan2k21 avatar Sep 11 '24 08:09 ayan2k21

Great! Thanks for the update.

ad1happy2go avatar Sep 26 '24 12:09 ad1happy2go