inlong
inlong copied to clipboard
[Feature][Manager] Add Apache Doris load node management
Description
Add Apache Doris Node Management, you can refer to: https://inlong.apache.org/docs/next/design_and_concept/how_to_extend_data_node_for_manager
Dashboard: https://inlong.apache.org/docs/next/design_and_concept/how_to_write_plugin_dashboard
Use case
No response
Are you willing to submit PR?
- [ ] Yes, I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Motivation
To surport the ability to Doris data integration, we need to add Apache Doris Load Node for management
Design
The design mainly follows the document Manager Plugin
- Add corresponding SinkType enumeration in enumeration type org.apache.inlong.manager.common.Enums.
- In org.apache.inlong.manager.common.Pojo.Sink,create folder path,create the corresponding entity class.
- In the org.Apache.Inlong.Manager.Service.Sink path, created under the corresponding tools
- Support data source to LoadNode conversion function, reference code org.Apache. Inlong.Manager.Service.Sort.Util.LoadNodeUtils
Implementation
- Add DORIS in enumeration type org.apache.inlong.manager.common.enums.SinkType
- Create folder "Doris" in org.apache.inlong.manager.common.pojo.sink, and create corresponding entity class:
- DorisSink
- DorisSinkDTO
- DorisSinkRequest
- DorisColumnInfo
- DorisTableInfo
- Create folder "doris" in org.apache.inlong.manager.service.sink and implement the class:
- DorisSinkOperator
- Add createLoadNode function in org.apache.inlong.manager.service.sort.util.LoadNodeUtils, it is like as follows:
public static DorisLoadNode createLoadNode(DorisSink dorisSink, List<FieldInfo> fieldInfos,
List<FieldRelation> fieldRelations, Map<String, String> properties){
\\TODO
}
@dockerzhang hi, can you assign this to me?
@Loveca LGTM
Duplicate with #6232, close this one.