[Improve][connector-v2] SourceReader Base implementation of One reader multiple splits

[ashulin]

Search before asking

• [X] I had searched in the feature and found no similar feature requirement.

Description

Most readers fall into one of the following categories:

1. One reader single splits. (Some dead simple blocking readers)

2. One reader multiple splits.

   1. Sequential Single Split (File, database query, most bounded splits)
   2. Multi-split multiplexed (Kafka, Pulsar, Pravega, ...)
   3. Multi-split multi-threaded (Kinesis, ...)

For case 1, we already have the base implementation.

For case 2, we don't have a uniform implementation. For example, for kafka and pulsar source, their implementations are different, which makes code quality and code review difficult

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[lhyundeadsoul]

I am curious why there is always one reader rather than multi-reader & multi-split .

[ashulin]

@lhyundeadsoul Because each degree of parallelism corresponds to one reader, and readers are isolated from each other, we only discuss based on one reader;

[ashulin]

@hailin0 The multi-split multiplexed feature of the CDC reader can be tracked using this issue.

[hailin0]

@ashulin please assign to me

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.