django-celery-results icon indicating copy to clipboard operation
django-celery-results copied to clipboard

SynchronousOnlyOperation occurs when using django-celery-results with gevent/async workers

Open JaeHyuckSa opened this issue 1 month ago • 0 comments

When running Celery with -P gevent and django-celery-results as the result backend, Django raises SynchronousOnlyOperation: You cannot call this from an async context whenever Celery tries to store task state (STARTED, etc.).

uv run --no-sync celery -A config.settings.components.celery worker
      -P gevent -c 256 -Q io
      -O fair --prefetch-multiplier=1
      --loglevel=info
      -n io@%h
/opt/venv/lib/python3.13/site-packages/celery/app/trace.py:662: RuntimeWarning: Exception raised outside body: SynchronousOnlyOperation('You cannot call this from an async context - use a thread or sync_to_async.'):
Traceback (most recent call last):
  File "/opt/venv/lib/python3.13/site-packages/celery/app/trace.py", line 443, in trace_task
    task.backend.store_result(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        uuid, {'pid': pid, 'hostname': hostname}, STARTED,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        request=task_request,
        ^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/venv/lib/python3.13/site-packages/celery/backends/base.py", line 526, in store_result
    self._store_result(task_id, result, state, traceback,
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       request=request, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/django_celery_results/backends/database.py", line 151, in _store_result
    self.TaskModel._default_manager.store_result(**task_props)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/django_celery_results/managers.py", line 42, in _inner
    return fun(*args, **kwargs)
  File "/opt/venv/lib/python3.13/site-packages/django_celery_results/managers.py", line 167, in store_result
    obj, created = self.using(using).get_or_create(task_id=task_id,
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
                                                   defaults=fields)
                                                   ^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 948, in get_or_create
    return self.get(**kwargs), False
           ~~~~~~~~^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 631, in get
    num = len(clone)
  File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 368, in __len__
    self._fetch_all()
    ~~~~~~~~~~~~~~~^^
  File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 1954, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/django/db/models/query.py", line 93, in __iter__
    results = compiler.execute_sql(
        chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size
    )
  File "/opt/venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py", line 1621, in execute_sql
    cursor = self.connection.cursor()
  File "/opt/venv/lib/python3.13/site-packages/django/utils/asyncio.py", line 24, in inner
    raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

  warn(RuntimeWarning(

Environment:

  • Django 5.2.x
  • django-celery-results 2.6.0
  • Celery 5.5.x (gevent pool)
  • Python 3.13
  • db: postgresql

JaeHyuckSa avatar Nov 26 '25 06:11 JaeHyuckSa