procrastinate-org
RuntimeError: There is no current event loop in thread 'Thread-10 (_handle_event_internal)'.
I am using procrastinate in a synchronous way in my Flask server with Flask-SocketIO.
Even though procrastinate.manager.JobManager.list_jobs is a synchronous method, calling it like below
app = App(
connector=Psycopg2Connector(
host=host,
user=user,
password=password,
)
)
job_manager = JobManager(app)
@socketio.on("message")
def handle_message(data):
job_id = sometask.defer(data=data)
jobs = job_manager.list_jobs(job_id)
gives the error
Exception in thread Thread-10 (_handle_event_internal):
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/home/gameveloster/foo/backend/venv/lib/python3.10/site-packages/socketio/server.py", line 733, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, *data[1:])
File "/home/gameveloster/foo/backend/venv/lib/python3.10/site-packages/socketio/server.py", line 758, in _trigger_event
return self.handlers[namespace][event](*args)
File "/home/gameveloster/foo/backend/venv/lib/python3.10/site-packages/flask_socketio/__init__.py", line 282, in _handler
return self._handle_event(handler, message, namespace, sid,
File "/home/gameveloster/foo/backend/venv/lib/python3.10/site-packages/flask_socketio/__init__.py", line 828, in _handle_event
ret = handler(*args)
File "/home/gameveloster/foo/backend/backend/app/__init__.py", line 53, in handle_message
jobs = job_manager.list_jobs()
File "/home/gameveloster/foo/backend/venv/lib/python3.10/site-packages/procrastinate/utils.py", line 149, in wrapper
return sync_await(awaitable=awaitable)
File "/home/gameveloster/foo/backend/venv/lib/python3.10/site-packages/procrastinate/utils.py", line 196, in sync_await
loop = asyncio.get_event_loop()
File "/usr/lib/python3.10/asyncio/events.py", line 656, in get_event_loop
raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Thread-10 (_handle_event_internal)'.
/usr/lib/python3.10/threading.py:1018: RuntimeWarning: coroutine 'JobManager.list_jobs_async' was never awaited
self._invoke_excepthook(self)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
From the traceback, it looks like there was an attempt to call the async method JobManager.list_jobs_async which likely involves an event loop? I am unsure why calling a sync method involve calling an async method? Seems like the async method should wrap the sync method and not the other way around.
Note that the sync method sometask.defer managed to run successfully.
Will appreciate some help to get the sync method JobManager.list_jobs working in my sync environment, thank you!
Procrastinate is async at core.
We know that there are issues in the way this is handled at the moment and I'm not a huge fan of the current state of the lib at the moment, but I haven't really set aside some time to rework that
If you're interested in taking a bit of time to explore a trasition to a proper dual sync/async compat, feel free to have a look, I suggest starting from my PR linked above.
