inlong
inlong copied to clipboard
[INLONG-5460][Sort][Manager] Support Apache Hudi
Prepare a Pull Request
- Fixes #5099
Motivation
Support Apache Hudi in sort module and manager module
Modifications
- Extend a new Extract Node for Apache Hudi
- Extend a new Load Node for Apache Hudi
- Add the corresponding flink connectors for Apache Hudi
- Extend Extract Node and Load Node in manager module for apache Hudi
Verifying this change
- [x] This change added tests and can be verified as follows:
- add the Unit test of HudiLoadNode HudiLoadNodeTest
- add the Unit test of HudiExtractNode HudiExtractNodeTest
- add Hudi source service test HudiSourceServiceTest for manager module
- add Hudi sink service test HudiSinkServiceTest for manager module
@Jellal-HT great job. please add the license file for the new dependency, and you can refer to https://inlong.apache.org/zh-CN/community/how-to-maintain-3rd-party-dependencies/.
otherwise, the workflow runs fail. Please check again.
hi @Jellal-HT A small suggested version upgrade to 0.12 for hudi.
Are you missing the step of bundling extract node and load node into the InLong sort ?
duplicated with https://github.com/apache/inlong/pull/6789 and https://github.com/apache/inlong/pull/6790, close it.