roadrunner
roadrunner copied to clipboard
[💡FEATURE REQUEST]: Adopt CFS algorithm (or other algo from this family) to the PQ
At the moment, if a user sends a lot of jobs (with slow workers) to the pipeline with high priority, other jobs, with low priority seem to be in the PQ for a long time. For example, imagine, that the user sends 10 jobs per second with priority 1. At the same time, other driver reads the different type of jobs with priority 10. PQ would look like the following:
Priority 1 -- [job] [job] [job] [job] [job] ... [job-1000]Priority 10 -- [job] [job] [job] [job][job] ... [job-1000]
Workers will consume only 1. When priority 1 will be consumed, only after that workers can start priority 10.
We need smt like fair scheduler here. So, every priority will have their time quants to consume the jobs.