[HUDI-4601] read error from MOR table after compaction

[wuwenchi]

Change Logs

Describe context and summary for this change. Highlight if any code was copied.

Impact

Describe any public API or user-facing feature change or any performance impact.

Risk level: none | low | medium | high

Choose one. If medium or high, explain what verification was done to mitigate the risks.

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[wuwenchi]

@danny0405 can you help review? thanks!

[danny0405]

Thanks, can you describe a little what are we trying to fix here ?

[wuwenchi]

@danny0405 The current partition key is stored in partSpec. If it is a partition key, the partition value can be used directly without reading the underlying data file. But when timestamp is used as the partition key, the partition value is not the real value, because the partition value is converted according to the real value, such as yyyy-mm-dd. So, it is necessary to remove the timestamp type in the partSpec.

[danny0405]

Thanks so much for the contribution @wuwenchi , i have reviewed and applied a patch: 4601.patch.zip

You can apply the patch and force-push the branch then :)

[wuwenchi]

Thank you very much for your detailed and patient review! @danny0405

[hudi-bot]

CI report:

• 800e7efa50a08471f7964b68b7ceff28016f17d0 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build