seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Feature][Zeta] Support Parallelism Inference

Open xiaochen-zhou opened this issue 1 month ago • 1 comments

Purpose of this pull request

Supports automatic parallelism inference for sources that implement the SupportParallelismInference interface (e.g., Paimon connector). When enabled, the engine will automatically determine the optimal parallelism based on data characteristics instead of using default parallelism value: 1

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Add new tests

Check list

  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
  • [ ] If necessary, please update incompatible-changes.md to describe the incompatibility caused by this PR.
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update plugin-mapping.properties and add new connector information in it
    2. Update the pom file of seatunnel-dist
    3. Add ci label in label-scope-conf
    4. Add e2e testcase in seatunnel-e2e
    5. Update connector plugin_config

xiaochen-zhou avatar Nov 23 '25 05:11 xiaochen-zhou

I have updated the code based on the review suggestions. Please take a look when you have time. @zhangshenghang

xiaochen-zhou avatar Nov 25 '25 13:11 xiaochen-zhou