eventmesh icon indicating copy to clipboard operation
eventmesh copied to clipboard

[ISSUE #5108] Abstracting and transforming EventMeshFunction, and implementing FunctionRuntime.

Open cnzakii opened this issue 5 months ago • 0 comments

Fixes #5108

Motivation

To adapt to the V2 architecture, the EventMeshFunction section needs to be upgraded and transformed. This includes redefining EventMeshFunction, making adaptive modifications to the existing filter and transformer functionalities, and reserving a unified interface for potential new Function features in the future. Based on this, the code for FunctionRuntime will be completed.

Modifications

For the Function module:

  1. A new eventmesh-function module has been added, consolidating the api module and the original filter and transformer modules for easier management.
  2. The EventMeshFunction interface has been defined with reference to java.util.function.Function. The data in Eventmesh is processed through the apply method of the function.
  3. The AbstractEventMeshFunctionChain subclass has been abstracted from EventMeshFunction, defining additional methods for chain processing, such as addFirst, addLast, etc.
  4. The original Transformer interface now inherits from and implements the EventMeshFunction interface, including the apply method for compatibility.

For the FunctionRuntime module:

  1. Referring to ConnectorRuntime, it implements connecting to adminServer, reporting heartbeat and current status to the adminServer, obtaining configuration information from the adminServer, and initializing the connectorService.
  2. A function processing chain is constructed based on the FunctionConfigs in the configuration and is used to process data in ConnectRecord.

Supplement:

Regarding FunctionConfigs in FunctionRuntime, the related configurations and functionalities are as follows:

  1. Users can configure any number of EventMeshFunction in FunctionConfigs according to their needs:

    functionConfigs:
      # filter
      - functionType: filter
        condition:
          source:
            - prefix: "eventmesh."
      # transformer
      - functionType: transformer
        transformerType: template
        valueMap:
          data-name: "$.data.name"
          data-num: "$.data.num"
          event-type: "$.type"
        template: "{\"type\":\"${event-type}\",\"name\":\"${data-name}\",\"no\":${data-num}}"
      # filter
      - functionType: filter
        condition:
          type:
            - suffix: "put"
      # transformer
      - functionType: transformer
        transformerType: constant
        content: "{\"name\":\"jack\",\"no\":123}"
    

    In the above EventMeshFunction, besides specifying the functionType, other fields are the same as those required by the original Function. For example, the filter requires condition information, which only needs to be added in the YAML file format.

  2. The EventMeshFunction in FunctionConfigs will be built into a function processing chain in sequence:

    input -> filter -> transformer -> filter -> transformer -> output
    

    The data entering the FunctionRuntime will flow along the route above. If the data is filtered, the processing will exit directly.

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

cnzakii avatar Sep 24 '24 14:09 cnzakii