arteria
Setting to turn off atomic transactions
The methods run_task() and run_next_task() are both annotated @atomic. I wonder what the reason for this is.
Django has a setting ATOMIC_REQUESTS for tying transactions to HTTP requests. Could we have a similar setting to control whether background tasks are executed within an atomic transaction or not?
This could be useful for cases where people are forced to use SQLite, but want to minimize the time the database gets locked. Currently, the atomic transaction locks the database for the entire task duration even if only read-only transactions are executed. This can be problematic for long-running tasks that do network requests for example.
It would also give a task author the opportunity to control which parts of their tasks should be run within a transaction. Currently, you are forced into one large transaction.
PS. Yes, I am aware that SQLite does not support concurrency and that this is a known issue.
Sorry but i don't know the reason. We forked this app from https://github.com/lilspikey/django-background-task and to be honest never thought about this part of the code.
I think a setting is a good idea. I will take a look ATOMIC_REQUESTS but i'm quite busy at the moment. Maybe someone can help out.
For other people who might be interested in this:
I am currently using a nasty hack that uses its own DBTaskRunner to prevent tasks from executing in large atomic transactions. It uses __wrapped__() to escape the @atomic annotation.
Having a proper solution here upstream with a setting is preferable though.
How about we just drop the atomic decorator completely until an option is created? This is more confusing than helpful, especially since the documentation doesn't mention either behavior at all.
@walterrenner @jvamvas what do you think regarding https://github.com/arteria/django-background-tasks/issues/113#issuecomment-345478320 ?
@philippeowagner i don't see any side effects when removing the decorator.
I just removed the @atomic decorators and ran the tox test suite. Every test has passed. So removing it completely is fine for me.