django-q2
django-q2 copied to clipboard
Running multiple workers with multiple clusters
Hey, I am working on a video editing project. Sometimes video processing can take a lot of time. So I want to run multiple tasks at same time.
What I have done - Right now I am using single cluster and only one task is executed at one time. So is it possible to run multiple clusters on same machine and have multiple tasks running at same time ? Thanks
You can run multiple clusters: https://django-q2.readthedocs.io/en/master/cluster.html#multiple-clusters, how you do this depends on how you are deploying it. If you are running containers, then you can spin up a second container for the management command.
You can also set the amount of workers within your cluster: https://django-q2.readthedocs.io/en/master/configure.html#workers