inlong icon indicating copy to clipboard operation
inlong copied to clipboard

[INLONG-5043][Manager] Add Apache Doris load node management

Open Loveca opened this issue 3 years ago • 0 comments

Prepare a Pull Request

Motivation

To surport the ability to Doris data integration, we need to add Apache Doris Load Node for management

Design

The design mainly follows the document Manager Plugin

  1. Add corresponding SinkType enumeration in enumeration type org.apache.inlong.manager.common.Enums.
  2. In org.apache.inlong.manager.common.Pojo.Sink,create folder path,create the corresponding entity class.
  3. In the org.Apache.Inlong.Manager.Service.Sink path, created under the corresponding tools
  4. Support data source to LoadNode conversion function, reference code org.Apache. Inlong.Manager.Service.Sort.Util.LoadNodeUtils

Implementation

  1. Add DORIS in enumeration type org.apache.inlong.manager.common.enums.SinkType
  2. Create folder "Doris" in org.apache.inlong.manager.common.pojo.sink, and create corresponding entity class:
  • DorisSink
  • DorisSinkDTO
  • DorisSinkRequest
  • DorisColumnInfo
  • DorisTableInfo
  1. Create folder "doris" in org.apache.inlong.manager.service.sink and implement the class:
  • DorisSinkOperator
  1. Add createLoadNode function in org.apache.inlong.manager.service.sort.util.LoadNodeUtils, it is like as follows:
public static DorisLoadNode createLoadNode(DorisSink dorisSink, List<FieldInfo> fieldInfos,
            List<FieldRelation> fieldRelations, Map<String, String> properties){
                 \\TODO 
      }

Loveca avatar Aug 17 '22 07:08 Loveca