[HUDI-7718] Try to fetch the latestSourceProfile in HoodieIncrSource

[vinishjail97]

Change Logs

Follow-up PR for https://github.com/apache/hudi/pull/11159 Try to fetch the latestSourceProfile always, this ensures the profile is refreshed if it's no longer valid. The implementation of source profile takes care when a profile needs to be refreshed or re-computed again.

Impact

A new constructor added for HoodieIncrSource for emitting metrics related to source parallelism and source bytes ingested.

Risk level (write none, low medium or high below)

Medium

Documentation Update

None.

Contributor's checklist

• [x] Read through contributor's guide
• [x] Change Logs and Impact were stated clearly
• [x] Adequate tests were added if applicable
• [x] CI passed

[hudi-bot]

CI report:

• 1f7af25d84c5261699be693b49687e4dbdd266af Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build