django-tasks
django-tasks copied to clipboard
RealOrangeOne
Respect result TTL on RQ backend
Right now, there doesn't appear to be any way to tell tasks to retain their results for a non-default period of time. When using the RQ backend, that period is 500 seconds, which is far too short for my team's needs.
When using django-rq's @job, you can specify a result_ttl keyword arg to the decorator, but that kwarg isn't available in django-tasks' @task. You also can't seem to set result_ttl via the enqueue() call.
You can specify a DEFAULT_RESULT_TTL in either the RQ settings dict or in entries in RQ_QUEUES (settings from django-rq), but django-tasks ignores it.
You can specify a
DEFAULT_RESULT_TTLin either theRQsettings dict or in entries inRQ_QUEUES(settings from django-rq), but django-tasks ignores it.
This sounds like the core of the issue. There currently isn't an API for doing backend-specific modifiations to a task (because creating an API that generic would be incredibly difficult).
However, django-tasks should absolutely be respecting the existing setting. It's not doing anything complex around Job creation, so I'm surprised these are getting missed.
This is likely because RQ.DEFAULT_RESULT_TTL is only used by django-rq's @job decorator, and RQ_QUEUES.<queue_name>. DEFAULT_RESULT_TTL is only used by the DjangoRQ queue subclass.
django-tasks doesn't use @job, though I'm not really sure why the DjangoRQ one doesn't seem to get used. I believe that queue subclass is used by default, and AUTOCOMMIT is also True by default.
Hi, I'm the maintainer/author of both RQ and django-rq and I happened to stumble into this repo. Let me know if you need any APIs that will make integrating RQ with django-tasks easier and I'll try to accommodate them.
When you open an issue, feel free to ping me so I receive notifications in my email inbox.
This is likely because RQ.DEFAULT_RESULT_TTL is only used by django-rq's @job decorator, and RQ_QUEUES.<queue_name>. DEFAULT_RESULT_TTL is only used by the DjangoRQ queue subclass.
My reading of this is that it's more of an issue with django-rq than django-tasks, correct?
It's been a bit since I last messed with this (my team found that RQ did not make for a reliable enough backend for cron tasks, and switched back to Celery), but I think something django-tasks could do to remedy this issue is to better integrate with django_rq.queues.DjangoRQ, since it respects the individual queue definitions' DEFAULT_RESULT_TTL values.
django_rq.queues.DjangoRQ is the queue being used, so I think that integration is already being done.
I'm going to close this without more information.