cylc-flow
cylc-flow copied to clipboard
Event-driven task triggering
Post SoD we could easily do event-driven triggering (like we already do for event-driven finished-task removal, for instance): when a parent-task output is completed and the dependent child spawned, immediately check if the child's prerequisites are satisfied and if so immediately queue for submission. This would avoid another task pool iteration loop (find all tasks that are ready, and queue them then).
However I held off doing this during SoD implementation because I thought it would probably break our job submission batching which is important for large suites with task families whose members all become ready to run at once.
Internal queues probably provide the answer: event-driven queuing of satisfied tasks, then batch-submission of currently queued tasks. However, internal queues are not currently implemented as actual queues - they're just dicts that record information about tasks that actually remain in the main task pool. Re-implement queues as queues?