prefect icon indicating copy to clipboard operation
prefect copied to clipboard

[New Engine] Add flow run pause handling

Open desertaxle opened this issue 1 year ago • 3 comments

Adds handling for paused flow run to the new engine. Changes to the new engine are relatively small, and this PR mostly contains tests for the new verifying that pausing flow runs works as expected.

Example

Checklist

  • [ ] This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • [x] If this pull request adds new functionality, it includes unit tests that cover the changes
  • [x] This pull request includes a label categorizing the change e.g. maintenance, fix, feature, enhancement, docs.

For documentation changes:

  • [ ] This pull request includes redirect settings in netlify.toml for files that are removed or renamed.

For new functions or classes in the Python SDK:

  • [ ] This pull request includes helpful docstrings.
  • [ ] If a new Python file was added, this pull request contains a stub page in the Python SDK docs and an entry in mkdocs.yml navigation.

desertaxle avatar May 06 '24 20:05 desertaxle

Deploy Preview for prefect-docs-preview ready!

Name Link
Latest commit a1d0490cd3614cba3a2a9e14ed89a854ded5020c
Latest deploy log https://app.netlify.com/sites/prefect-docs-preview/deploys/6642b445cce6bf000873ad8a
Deploy Preview https://deploy-preview-13264--prefect-docs-preview.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar May 06 '24 20:05 netlify[bot]

I think a (more?) common pattern is pausing from within the body of a flow - that seems like it would require analogous changes to the new flow run engine yea?

AFAICT, pause_flow_run is blocking and doesn't require any changes to the flow engine to keep the flow from executing. suspend_flow_run raises a Pause signal, which bubbles up through the flow run engine and is caught by the entry point we use for flow runs from a deployment.

The changes in the task engine prevent a task from starting if the parent flow is paused, which can happen if the task is submitted to remote infrastructure. That hasn't been needed for flow runs because they have historically been blocking, but I could see that changing with the new engine. We would also need an orchestration rule to prevent starting flow runs from a paused parent run, so I think that might be best tackled in another PR.

desertaxle avatar May 07 '24 20:05 desertaxle

@desertaxle ah wow, that's nice! Thanks for explaining

cicdw avatar May 07 '24 20:05 cicdw