[HUDI-4781] Allow omit metadata fields for hive sync

[parisni]

trafficstars

Impact

This adds a new config hoodie.datasource.hive_sync.omit_metadata_fields, which default as previous behavoir. WHen true, this won't create the metadata fields in the hive table, and hide them for end users

**Risk level: low **

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[parisni]

Yep. Also tested with Athena ans spectrum

On September 6, 2022 1:43:35 AM UTC, Y Ethan Guo @.> wrote: @. commented on this pull request.

@.*** Have you tested if the Hudi table can be queried with Hive, if the Hudi meta fields are not synced?

@@ -103,6 +103,11 @@ public class HiveSyncConfigHolder { .key("hoodie.datasource.hive_sync.create_managed_table") .defaultValue(false) .withDocumentation("Whether to sync the table as managed table.");

• public static final ConfigProperty<Boolean> HIVE_OMIT_METADATA_FIELDS = ConfigProperty

•  .key("hoodie.datasource.hive_sync.omit_metadata_fields")
  

•  .defaultValue(false)
  

•  .sinceVersion("0.12.1")
  

We don't introduce new configs in minor releases. This should be 0.13.0 which is the next major version.

-- Reply to this email directly or view it on GitHub: https://github.com/apache/hudi/pull/6471#pullrequestreview-1096841811 You are receiving this because you were mentioned.

Message ID: @.***>

[hudi-bot]

CI report:

• f06264ba800c37b82240d0d0ddab82e182b7da6f UNKNOWN
• 071adff869adf398f19ef0a1e77fcbc9b0baa7d3 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build