texera icon indicating copy to clipboard operation
texera copied to clipboard

Migrate scala control messages to protobuf

Open shengquan-ni opened this issue 4 months ago • 0 comments

This PR migrates all Scala control messages and their corresponding handlers to use gRPC.


Main Changes

  1. Rewrote AsyncRPCServer and AsyncRPCClient

    • Adjusted both to fit into the generated gRPC interface, while keeping RPCs sent and received through actor messages.
  2. Centralized RPC Definitions

    • All RPC messages and method definitions are now located in:
      src/main/protobuf/edu/uci/ics/amber/engine/architecture/rpc
      
  3. Preserved Existing Handler Definitions

    • The previous way of defining RPC handlers has been retained for compatibility.
  4. Removed Unused RPCs

    • Deleted obsolete RPCs:
      • QueryCurrentInputTuple
      • ShutdownDPThread
  5. Unified Logic Modification RPCs

    • Merged Fries/Python logic modification RPCs into one:
      • Before:
        • Reconfigure (Fries)
        • UpdateExecutor (Python code update)
      • After:
        • ModifyLogic (Unified)

Steps to Create New RPCs

  1. Create a request proto message in controlcommands.proto with all necessary inputs.
  2. Create a response proto message in controlreturns.proto.
  3. Define the RPC:
    • For controller-handled RPCs, add the definition to controllerservice.proto.
    • For worker-handled RPCs, add it to workerservice.proto.
  4. Generate Scala code by running protocGenerate in the SBT shell.
  5. Implement the RPC handler in either ControllerAsyncRPCHandlerInitializer or WorkerAsyncRPCHandlerInitializer, and move it to the promisehandlers folder for better organization.

Changes in Sending RPCs

Example: Using the StartWorkflow RPC, which is handled by the controller.

  • Before this PR:

    send(StartWorkflow(), CONTROLLER).map { ret =>
      // handle the return value
    }
    
  • After this PR:

    controllerInterface.startWorkflow(EmptyRequest(), mkContext(CONTROLLER)).map { resp =>
        // handle the response
    }
    

Advanced Usage: Direct Control Invocation

To create a control invocation directly (advanced use case):

import edu.uci.ics.amber.engine.common.rpc.AsyncRPCClient.ControlInvocation

ControlInvocation(
METHOD_RPC_NAME,
PayloadRequest(),
AsyncRPCContext(senderID, receiverID),
commandID
)

shengquan-ni avatar Oct 17 '24 10:10 shengquan-ni