[HUDI-4734] Deltastreamer table config change validation

[jonvex]

Change Logs

Table configuration properties were not being validated in deltastreamer so changes to properties that shouldn't change were being allowed. To fix this, we now call the validator that is used by spark sql writer.

Impact

Now, a user should not be able to change immutable configs for a table which will silently cause issues down the line.

**Risk level: low **

Contributor's checklist

• [x] Read through contributor's guide
• [x] Change Logs and Impact were stated clearly
• [x] Adequate tests were added if applicable
• [x] CI passed

[nsivabalan]

have pushed out a commit to address minor comments.

[codope]

simply rebased as the flaky test was fixed recently

[nsivabalan]

I triaged as to why payloadClassUpdate test in deltastreamer was succeeding w/ this patch. I was expecting it to fail. apparently payload class is a top level config in deltastreamer and hence its not passed via regular props. So, it bypasses the validation we have. for now, lets proceed w/ the patch. later we can decide if we really need to add validation for payload class update as well.

also, have reverted m1 pom changes. once CI is green, we are good to land.

[hudi-bot]

CI report:

• 722ee61ed665a6e31cc4871bfd71c17f22673868 Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[nsivabalan]

azure CI failed due to flaky flink IT failure. going ahead w/ merge as we have code freeze coming up. changes in the patch is safe to land.