django-celery
django-celery copied to clipboard
beat and cam memory footprint reduction techniq (not an issue)
When beat or celerycam in running as separate processes ("manage.py celery beat" or "manage.py celery celerycam" executed) all django application is initialized and all dependent stuff loaded in memory.
In my case, my django app takes about 70-120Mb private memory on start and because of it, on production servers, beat and cam processes take the same amount of memory as regular celery worker or uwsgi process. That's about 200Mb in my case for beat+cam.
I wrote two scripts that run beat and cam with minimum dependencies: djcelery as installed app, database connection from main app config. At now beat+cam processes take only 64Mb (by 32Mb on one process).
Maybe my results will be useful to someone with memory optimization.