Running manage.py worker in a Cpanel shared env

[collinr3]

Hi,

Firstly, thank you for sharing this app - I heard Jamie mention it on DjangoChat a while back.

I'm now looking to use it and I wonder if you can help me.

I have my django app running in a shared env (Hostpresto) under Phusion passenger and I want to add 'django-db-queue' to manage periodic tasks. The hosting team suggest setting up a cron job to to run. manage.py worker periodically.

Q1: Do you have any experience of deploying this app into a cpanel /shared env, if so how did you make it work?

Q2: In your opinion, is using cron to run manage.py worker a reasonable approach.

Thoughts?

Rob

[RealOrangeOne]

Cron and manage.py worker aren't really compatible models. The worker command will never exit, as it waits for command to run.

Cpanel hosting of this is probably out of scope for an issue here, but if there's a way to run some form of background process, that's the way.

Alternatively, perhaps a manage.py worker --oneshot variant which runs all available jobs and then exits could be useful?

[j4mie]

Thanks for trying out django-db-queue!

As @RealOrangeOne says above, running the worker under cron doesn't really fit with django-db-queue's design, because cron is intended for finite tasks that do their work and then exit, whereas the worker is supposed to keep running in the background forever. You may be able to get this to work by doing something clever with pidfiles and a pile of bash, but it would probably be a bit unpredictable.

Note this issue is really a duplicate of https://github.com/dabapps/django-db-queue/issues/16

I would be happy to review a PR that adds this functionality, although I'm not completely decided that it's a good thing to do, as it's not something that we would need so would be a maintenance burden. It would depend on how big the diff ends up being!

[collinr3]

Thanks both for your input, really helpful. I'm going to do a couple of experiments, which I will report back on in due course, for the record.

1. A 'do nothing' option ie start the worker running 'as is' and give it a job to report daily 'alive a live o' - that will give me a feel for how long the worker will run for in my specific hosted env. This will likely take a small number of weeks to perform.

2. Implement a '--cycles' (working title) concept that instructs the worker on how long their 'shift' is for. My thinking being is that this builds in a bit of future proofing for the user that wants more than 1 but less than infinity. I will strive for simple elegance to minimise any maintenance burden. If I think the change meets the maintenance requirement I will submit a PR for consideration.

[collinr3]

Hi, Pull request #55 is now available for your consideration

[collinr3]

Also addresses #16