n8n icon indicating copy to clipboard operation
n8n copied to clipboard

feat(core): Add basic execution concurrency limits in main mode

Open netroy opened this issue 1 year ago • 2 comments

This PR implements a very rudimentary concurrency limit in main mode, to be a bit more consistent with queue mode, and also to prevent the application from thrashing the event-loop when under high load.

Ticket: N8N-7052

Review / Merge checklist

  • [x] PR title and summary are descriptive
  • [ ] Docs updated or follow-up ticket created.
  • [ ] Tests included

netroy avatar Jan 26 '24 12:01 netroy

jumping back on this now.

netroy avatar Feb 08 '24 15:02 netroy

More ideas:

  • Prevent users from setting any concurrency limit lower than 1. As an exception, consider supporting -1 for unlimited concurrency, with a warning.
  • Add Public API endpoints to read queue size and update the production concurrency limit, so cloud can dynamically adjust it for heavy users in lower plans.
  • Priority queuing for selected workflows?
  • Warn on startup if concurrency limit is too high for system specs.
  • Add a timestamp to the queued item and on resuming warn if the time elapsed between enqueueing and resuming exceeded a threshold.

ivov avatar Feb 09 '24 08:02 ivov

closing in favor of https://github.com/n8n-io/n8n/pull/9453

netroy avatar Jun 03 '24 17:06 netroy