flow-framework icon indicating copy to clipboard operation
flow-framework copied to clipboard

[REFACTOR] Improve Workflow State Resource updates

Open dbwiddis opened this issue 1 year ago • 1 comments

Is your feature request related to a problem?

Currently the Workflow State resource document is only additive, using a Painless script during provisioning to append to a list. This prevented iteratively updating it during deprovisioning, leading to open questions like #691.

Following #778 we will have the ability to update any individual element of the Workflow State which will bring these benefits:

  • Allow for incremental addition to the resources created without using Painless, improving portability
  • Allow for incremental removal from the resources created during deprovisioning, allowing the full deprovisioning to depend on this update. (See #780)
  • Allow for the easy removal of the "error" field on subsequent successful provisioning

What solution would you like?

Replace the painless script with appropriate code from #778. Use ifSeqNo() and ifPrimaryTerm() on the update requests to enforce concurrency, and retry if it fails.

dbwiddis avatar Jul 12 '24 18:07 dbwiddis

Coming from https://github.com/opensearch-project/flow-framework/pull/763#discussion_r1678660003, there's a lot of repetitive boilerplate used in Workflow Steps for exception handling during resource updates. As part of this refactoring, the "failed" update code should be de-duplicated.

dbwiddis avatar Jul 16 '24 15:07 dbwiddis