iwf icon indicating copy to clipboard operation
iwf copied to clipboard

V2 of waitForStateCompletion API using synchronous update feature

Open longquanzheng opened this issue 2 years ago • 1 comments

The current implementation uses a sub workflow: https://github.com/indeedeng/iwf/pull/345 requires to provide WaitingForCompletionStateExecutionIds on starting workflow which is cumbersome to use. It's required because we want minimize the initiation of these sub workflows as optimization.

Additionally, it's not efficient because of the overhead of starting/signaling another workflow. It consume quite a lot of actions

A better way is:

  • Extend the stateExecutionCounter to track the completed state executionId
  • Use synchronous update with workflow.Await( ()-> stateExecutionCounter.stateExecuctionCompleted(stateExeId) ) in the handler to wait for the execution

Note that continueAsNew need to be handled properly:

  • In validator, it should check if the workflow has met continueAsNew threshold. If so, then return error
  • In the workflow.Await(...) should also check continueAsNew threshold to return error so that workflow can do a continueAsNew
  • The API service need to retry when continueAsNew interrupts the waiting
  • returning state completion output will not be supported

Also, it doesn't allow reuse workflowId for different executions, because of the RunID issues in Temporal. This v2 can solve the problems

~~Note that sync update feature is not prod ready in Temporal yet. We will start working on it when it is ready.~~

longquanzheng avatar Sep 18 '23 21:09 longquanzheng

Above solution is really complicated to implement, an easy workaround for reusing workflowId is https://github.com/indeedeng/iwf/issues/404

longquanzheng avatar Jul 26 '24 17:07 longquanzheng