seatunnel
seatunnel copied to clipboard
apache
[Improvement][seatunnel-server] Improve the scheduler module to support more scheduling plugins
Purpose of this pull request
Improve the scheduler module to support more scheduling plugins
-
From the perspective of the code module structure, SeaTunnel is to support scheduler plug-in, and can support more scheduling plug-ins, such as dolphinscheduler, azkaban
-
The current code does not really implement scheduler plug-in
Configuration file:
The key of the current property is ds.* . If azkaban scheduling is added, what should this key be?
https://github.com/apache/incubator-seatunnel/pull/2602/files#diff-be5fe744bda2308d461ce12ee32f8cc115dac0682230814f2b96e6ee0f5ade27L29-R43
Plugin implementation:
At present, spring dependency injection is used. When adding a plug-in module, an interface has two implementation classes, and the project will report an error at this time.
I introduced the ConditionalOnProperty annotation. According to the value of the configuration file seatunnel.scheduler.type, which implementation class should be injected into spring, only one implementation class will take effect in the project.
https://github.com/apache/incubator-seatunnel/pull/2602/files#diff-b0fc519f508b3019a859edab06e94430b51bd2df0b7c8cd834076d97f07155efR32-R33
Check list
- [x] Code changed are covered with tests, or it does not need tests for reason:
- [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
- [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
Can you describe specifically what this PR does?
Improve the scheduler module to support more scheduling plugins
-
From the perspective of the code module structure, SeaTunnel is to support scheduler plug-in, and can support more scheduling plug-ins, such as dolphinscheduler, azkaban
-
The current code does not really implement scheduler plug-in
Configuration file:
The key of the current property is ds.* . If azkaban scheduling is added, what should this key be?
https://github.com/apache/incubator-seatunnel/pull/2602/files#diff-be5fe744bda2308d461ce12ee32f8cc115dac0682230814f2b96e6ee0f5ade27L29-R43
Plugin implementation:
At present, spring dependency injection is used. When adding a plug-in module, an interface has two implementation classes, and the project will report an error at this time.
I introduced the ConditionalOnProperty annotation. According to the value of the configuration file seatunnel.scheduler.type, which implementation class should be injected into spring, only one implementation class will take effect in the project.
https://github.com/apache/incubator-seatunnel/pull/2602/files#diff-b0fc519f508b3019a859edab06e94430b51bd2df0b7c8cd834076d97f07155efR32-R33
