django-db-queue
django-db-queue copied to clipboard
Add --drain flag to worker command
python manage.py worker --drain
will start a worker process which will drain all jobs in the queue (if there are any) and then exit. This addresses #16 (from seven years ago 😱) and #52, and is an alternative (and much simpler) solution to that proposed in #56.
Thanks @j4mie. I agree that it is simple and that there is attraction in that.
I'd previously gone for the more complex #56 --shift_limit
approach, specifically because of the potential issue @jordaneb raises. The dynamic between --rate
expected volumes of jobs, average job execution time and CRON interval is a complex one. The unknown is what undesirable side effects may be introduced into the data should there be multiple workers processing the same queue. That said, the issue cannot be easily mitigated in a CRON, environment, whatever the approach, so the 'warning' may be the most pragmatic.
One comment on the test for --drain
consider adding an extra step to submit a further Job after the worker has initially drained the queue to check that it doesn't get processed.