django-celery-results
django-celery-results copied to clipboard
SynchronousOnlyOperation occurs when using django-celery-results with gevent/async workers
When running Celery with -P gevent and django-celery-results as the result backend, Django raises SynchronousOnlyOperation: You cannot call this from an async context whenever Celery tries to store task state (STARTED, etc.).
uv run --no-sync celery -A config.settings.components.celery worker
-P gevent -c 256 -Q io
-O fair --prefetch-multiplier=1
--loglevel=info
-n io@%h
/opt/venv/lib/python3.13/site-packages/celery/app/trace.py:662: RuntimeWarning: Exception raised outside body: SynchronousOnlyOperation('You cannot call this from an async context - use a thread or sync_to_async.'):
Traceback (most recent call last):
File "/opt/venv/lib/python3.13/site-packages/celery/app/trace.py", line 443, in trace_task
task.backend.store_result(
~~~~~~~~~~~~~~~~~~~~~~~~~^
uuid, {'pid': pid, 'hostname': hostname}, STARTED,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
request=task_request,
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/opt/venv/lib/python3.13/site-packages/celery/backends/base.py", line 526, in store_result
self._store_result(task_id, result, state, traceback,
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
request=request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.13/site-packages/django_celery_results/backends/database.py", line 151, in _store_result
self.TaskModel._default_manager.store_result(**task_props)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.13/site-packages/django_celery_results/managers.py", line 42, in _inner
return fun(*args, **kwargs)
File "/opt/venv/lib/python3.13/site-packages/django_celery_results/managers.py", line 167, in store_result
obj, created = self.using(using).get_or_create(task_id=task_id,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
defaults=fields)
^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 948, in get_or_create
return self.get(**kwargs), False
~~~~~~~~^^^^^^^^^^
File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 631, in get
num = len(clone)
File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 368, in __len__
self._fetch_all()
~~~~~~~~~~~~~~~^^
File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 1954, in _fetch_all
self._result_cache = list(self._iterable_class(self))
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 93, in __iter__
results = compiler.execute_sql(
chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size
)
File "/opt/venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py", line 1621, in execute_sql
cursor = self.connection.cursor()
File "/opt/venv/lib/python3.13/site-packages/django/utils/asyncio.py", line 24, in inner
raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.
warn(RuntimeWarning(
Environment:
- Django 5.2.x
- django-celery-results 2.6.0
- Celery 5.5.x (gevent pool)
- Python 3.13
- db: postgresql