[HUDI-7501] Use source profile for S3 and GCS sources

[vinishjail97]

Change Logs

Use SourceProfile in S3/GCS sources for the sourceLimit and numPartitions. Instead of relying on the sourceLimit passed in from static config use the one provided by the source profile.

Impact

No impact to existing public constructors, the change is backwards compatible.

Risk level (write none, low medium or high below)

Medium

Documentation Update

None, this is just adding a new public constructor to include StreamContext as a parameter in S3/GCS sources.

Contributor's checklist

• [x] Read through contributor's guide
• [x] Change Logs and Impact were stated clearly
• [x] Adequate tests were added if applicable
• [ ] CI passed