[SPIP-3][Console] Introduce flink cdc pipeline
Search before asking
- [X] I had searched in the SPIP and found no similar SPIP.
Motivation
For example, SPIP-2 supports catalog management, and we also support flink cdc, allowing us to submit a flink cdc yaml file to complete the development of synchronous jobs. This has the benefit of fully supporting the table schema change and multi-table synchronization capabilities of flink cdc. Improve the overall product user experience
Design Detail
Part 1: The front-end changes are minimal, adding support for the job type, flink cdc, and adding support for flink cdc yaml in the sql edit box
Part 2: As with the current flink sql logic, yaml is compiled into a jar and submitted to the yarn/k8s cluster 1. app create add flink cdc type 2. createPipelineInstance main class flink cdc main 3. ApplicationActionServiceImp#getUserJarAndAppConf
Compatibility, Deprecation, and Migration Plan
No response
Test Plan
- e2e test for flink cdc
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
https://github.com/apache/incubator-streampark/issues/3857
Plan
- [Feature] Add CDC jar client for support cdc yaml #4163
- [Feature] Backend modifications support the cdc yaml api #4164
- [Feature] The front-end is modified to support the cdc yaml api #4165
- [Feature] E2E to support the cdc yaml api #4165
2.2 roadmap dont have this feature?