django-q2 icon indicating copy to clipboard operation
django-q2 copied to clipboard

Running multiple workers with multiple clusters

Open CommitSithLord opened this issue 2 years ago • 1 comments

Hey, I am working on a video editing project. Sometimes video processing can take a lot of time. So I want to run multiple tasks at same time.

What I have done - Right now I am using single cluster and only one task is executed at one time. So is it possible to run multiple clusters on same machine and have multiple tasks running at same time ? Thanks

CommitSithLord avatar Feb 16 '23 21:02 CommitSithLord

You can run multiple clusters: https://django-q2.readthedocs.io/en/master/cluster.html#multiple-clusters, how you do this depends on how you are deploying it. If you are running containers, then you can spin up a second container for the management command.

You can also set the amount of workers within your cluster: https://django-q2.readthedocs.io/en/master/configure.html#workers

GDay avatar Feb 16 '23 23:02 GDay