django-rq icon indicating copy to clipboard operation
django-rq copied to clipboard

Django testing: DB Connection closed after processing in burst mode,

Open pratyushsharma opened this issue 9 years ago • 1 comments

Add a option to persist db connections

get_worker('high').work(burst=True) obj = Model.objects.filter(user=self.test_user).first()


Python==3.4 django==1.9.7 django-rq==0.9.0 rq==0.5.6 rq-scheduler==0.6.0 redis=2.10.5


Traceback (most recent call last): File "/authentication/tests.py", line 21, in setUp self.assertEquals(len(User.objects.all()) > 1, True) File "/venv/lib/python3.4/site-packages/django/db/models/query.py", line 240, in len self._fetch_all() File "/venv/lib/python3.4/site-packages/django/db/models/query.py", line 1074, in _fetch_all self._result_cache = list(self.iterator()) File "/venv/lib/python3.4/site-packages/django/db/models/query.py", line 52, in iter results = compiler.execute_sql() File "/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 846, in execute_sql cursor = self.connection.cursor() File "/venv/lib/python3.4/site-packages/django/db/backends/base/base.py", line 233, in cursor cursor = self.make_cursor(self._cursor()) File "/venv/lib/python3.4/site-packages/django/db/backends/base/base.py", line 206, in _cursor return self.create_cursor() File "/venv/lib/python3.4/site-packages/django/db/utils.py", line 95, in exit six.reraise(dj_exc_type, dj_exc_value, traceback) File "/venv/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise raise value.with_traceback(tb) File "/venv/lib/python3.4/site-packages/django/db/backends/base/base.py", line 206, in _cursor return self.create_cursor() File "venv/lib/python3.4/site-packages/django/db/backends/postgresql/base.py", line 210, in create_cursor cursor = self.connection.cursor() django.db.utils.InterfaceError: connection already closed

pratyushsharma avatar Jun 28 '16 09:06 pratyushsharma

Django's ORM doesn't play well with code that calls os.fork() (see https://code.djangoproject.com/ticket/20562) . A workaround for this is to re-open the DB connection every time your function runs.

We can also patch DjangoRQ class such that it re-opens a new DB connection after os.fork() is called. I'd welcome a PR for this :)

selwin avatar Aug 21 '16 11:08 selwin