django-q icon indicating copy to clipboard operation
django-q copied to clipboard

deploy django_q to heroku

Open shriDeveloper opened this issue 3 years ago • 3 comments

I have been using django_q for my background processing. and i want to deploy it to heroku now.

Skimming through documentation , i did not found nay guidelines on how to deploy my app to heroku. (which uses django_q) .

Also, would like to know, what is the significance of "workers" in settings.py. How can i decide how many workers to add on a paid heroku plan.

shriDeveloper avatar Apr 24 '21 10:04 shriDeveloper

Workers signifies the amount of multiprocessing workers per cluster instance. For Heroku you would set up a procfile with a 'worker' line. The two should not be confused. The 'worker' can have multiple Dynos and each dyno then can have multiple Django-Q workers(through the worker config setting). The amount of workers you can run per dyno depends on the Dyno size and on the footprint of your application.

Here's an example Procfile:

web:  gunicorn -c gunicorn-conf.py myapp.wsgi:application
worker: python manage.py qcluster

Here is one of my personal setups on Heroku, with Gunicorn and Postgres. Using the Nginx and Pgbouncer buildpacks to buffer requests and making sure the workers don't drain the pg connections:

web: bin/start-nginx bin/start-pgbouncer gunicorn -c gunicorn-conf.py myapp.wsgi:application
worker: bin/start-pgbouncer python manage.py qcluster

with a gunicorn config that points to a unix socket:

def when_ready(server):
    open("/tmp/app-initialized", "w").close()


bind = "unix:///tmp/nginx.socket"
workers = 3
threads = 4
preload_app = True
timeout = 25
max_requests = 1000
max_requests_jitter = 100
keepalive = 4
capture_output = True

Koed00 avatar Apr 24 '21 11:04 Koed00

Thanks a ton. Will deploy using this. Any public repository you wanna share to see, what the overall file structures gonna be for Heroku.

shriDeveloper avatar Apr 24 '21 13:04 shriDeveloper

@shriDeveloper I was looking for help on this as well. Thank you. Seems that you have to also log into Heroku and "turn on" the extra Dyno that the procfile creates...?

https://i.imgur.com/jRDWi5I.png

mnoah66 avatar Apr 18 '22 14:04 mnoah66