texera icon indicating copy to clipboard operation
texera copied to clipboard

Introduce State Related Operators

Open aglinxinyuan opened this issue 5 months ago • 0 comments

This PR introduces State to Data and Data to State operators, enabling users to convert between data and state.

New Operator Groups and Constants:

  • Added STATE_GROUP to OperatorGroupConstants and updated the group hierarchy to include this new group.

New Operators:

  • Implemented DataToState Operator to convert data into state.
  • Implemented StateToData Operator to convert state back into data.

image

The State to Data operator retrieves the first tuple from the first port, converting it into a state while taking all tuples from the second port as data to pass downstream. The user can configure whether the state is passed once or to all downstream operators.

image

The Data to State operator retrieves the first state from the input port, converting it into a tuple and output to the first output port while taking all tuples from the input port as data and output to the second output port. The user has to define the state's scheme.

image

Even though this pair of operators will only take the first tuple/state, it's possible to nest multiple pairs of those operators to convert multiple tuples/states.

image

This pull request introduces a new feature for state management within the workflow system. The changes include adding new operator groups and implementing two new operators: DataToStateOpDesc and StateToDataOpDesc. These operators facilitate the conversion between data and state within workflows.

aglinxinyuan avatar Sep 21 '24 07:09 aglinxinyuan