[HUDI-8203] Make record merge mode the primary merging config

[jonvex]

Change Logs

Record Merge Mode is the primary way to configure how your merging logic works. Avro payload based merging is only used if you set a payload that is not the default payload or the overwrite with latest payload.

The filegroup reader now will use payload based merging when the table is configured to use a custom avro payload.

Fg reader now uses payload based merging if the user uses that on the write side. This leads to consistent results when reading vs compaction

Impact

Part of umbrella to simplify merge configs for 1.X

Risk level (write none, low medium or high below)

low

Documentation Update

N/A

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[jonvex]

https://github.com/apache/hudi/pull/11943#discussion_r1811096912 https://issues.apache.org/jira/browse/HUDI-8408

[hudi-bot]

CI report:

• afe0461766eb01a1e5cd35bca09aebe77d2ce7a2 UNKNOWN
• 12c70f8fb325b6e0e741fec7ee1f112f66272d61 UNKNOWN
• c1e2ca4da097740ce98be4ac2a890a6ceb861069 UNKNOWN
• 38d289c8fddb383514e7a790265bea6633ca3df2 UNKNOWN
• d4b1ca892f76a18228a5f78aad3bf20f67c74335 UNKNOWN
• aeb502a82e85dc00b9b1d64bce6a9305ef68173d Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build