prefect
prefect copied to clipboard
Add max concurrent submitted & running flow runs on local agent
Summary
Our use case is poorly served by having agents submit as many flows as they can. We limit the number that actually run concurrently on the agent but they are still running on the agent as far as prefect is concerned. This PR adds some checks so that the agent will leave flow runs in the scheduled state if the number that have been submitted on the agent but not finished grows greater than a configurable parameter.
Steps Taken to QA Changes
Code has been tested manually. The agent can be tested by running it with the parameter set and then running flow runs and monitoring their status. Better testing can be looked into after we get some initial comments.
Checklist
- [x] Closes https://github.com/PrefectHQ/prefect/issues/6231
This pull request is:
- [ ] A documentation / typographical error fix
- No tests or issue needed
- [ ] A short code fix
- Please reference the related issue by including "closes
<link to issue>" in this Pull Request's summary section.- If no issue exists, please create a bug report issue
- Please include tests. One-line fixes without tests will not be accepted unless it's related to the documentation only.
- Please reference the related issue by including "closes
- [x] A new feature implementation
- Please reference the related issue by including "closes
<link to issue>" in this Pull Request's summary section.- If no issue exists, please create a feature enhancement issue
- Please include tests
- Please make sure that your QA steps are both thorough and easy to reproduce by somebody with limited knowledge of the feature that you are submitting
- Please reference the related issue by including "closes
Happy engineering!