[HUDI-9526][DRAFT] Use HoodieFileGroupReader throughout the CDC flow

[the-other-tim-brown]

Change Logs

• Update spark reader to use HoodieFileGroupReader
• Add logic for easily checking if a row represents a delete operation to avoid converting to and from avro multiple times in the CDC spark reader
• TODO: compaction support

Impact

Describe any public API or user-facing feature change or any performance impact.

Risk level (write none, low medium or high below)

If medium or high, explain what verification was done to mitigate the risks.

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

• The config description must be updated if new configs are added or the default value of the configs are changed
• Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the instruction to make changes to the website.

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[vinothchandar]

@danny0405 Can you please list out your remaining concerns as a checklist..

[vinothchandar]

As discussed, I can shepherd this PR once @yihua chimes in.

[danny0405]

@danny0405 Can you please list out your remaining concerns as a checklist..

Left some comments.

[vinothchandar]

@jonvex you plan to review this?

[hudi-bot]

CI report:

• 15a27a8e0312a55fc7e536c4bf91d15edd3294a0 UNKNOWN
• 93425d152b8586d862d0f5fa4a58f69b12628986 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build