Some tests fail with RuntimeError: There is no current event loop in thread 'MainThread'

[rubik]

test_no_jobs and test_health_check_direct fail with this same error:

 pytest -k test_health_check_direct
Test session starts (platform: linux, Python 3.7.4, pytest 4.4.1, pytest-sugar 0.9.2)
rootdir: /home/miki/exp/arq, inifile: setup.cfg, testpaths: tests
plugins: mock-1.10.4, sugar-0.9.2, timeout-1.3.3, cov-2.6.1, aiohttp-0.3.0, toolbox-0.4
timeout: 5.0s
timeout method: signal
timeout func_only: False
collecting ... 

―――――――――――――――――――――――――――――――――――― test_health_check_direct ――――――――――――――――――――――――――――――――――――

loop = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def test_health_check_direct(loop):
        class Settings:
            pass
    
>       assert check_health(Settings) == 1

tests/test_worker.py:41: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
arq/worker.py:610: in check_health
    loop = asyncio.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7f080bc70210>

    def get_event_loop(self):
        """Get the event loop.
    
        This may be None or an instance of EventLoop.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                isinstance(threading.current_thread(), threading._MainThread)):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
            raise RuntimeError('There is no current event loop in thread %r.'
>                              % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/usr/lib64/python3.7/asyncio/events.py:644: RuntimeError

 tests/test_worker.py ⨯                                                            100% ██████████

Results (0.31s):
       1 failed
         - tests/test_worker.py:37 test_health_check_direct
      97 deselected

[samuelcolvin]

I think the problem is with a conflicting fixture loop provided by pytest-toolbox, I can remove that but to confirm the problem, please run:

pytest -p no:toolbox -k test_health_check_direct

[samuelcolvin]

@rubik, did the above fix this?

[rubik]

@samuelcolvin Yes, the test passes with that option.

[pratheekrebala]

FWIW - this also works for me.

[AngellusMortis]

I am getting the same error with pytest at all.

Traceback (most recent call last):
  File "/usr/local/bin/arq", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/arq/cli.py", line 54, in cli
    run_worker(worker_settings_, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/arq/worker.py", line 867, in run_worker
    worker = create_worker(settings_cls, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/arq/worker.py", line 863, in create_worker
    return Worker(**{**get_kwargs(settings_cls), **kwargs})  # type: ignore
  File "/usr/local/lib/python3.10/site-packages/arq/worker.py", line 262, in __init__
    self.loop = asyncio.get_event_loop()
  File "/usr/local/lib/python3.10/asyncio/events.py", line 656, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'MainThread'.

After doing some testing and debugging, it basically looks like if anything uses and closes an asyncio loop inside of the main thread before ARQ can initialize, it causes the worker to throw the error.

Reproduction code with even using ARQ:

import asyncio

async def test():
     print("test")
 
asyncio.run(test())
asyncio.get_event_loop()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/asyncio/events.py", line 656, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'MainThread'.

ARQ should probably be doing something like the following instead of calling get_event_loop directly and create/maintain its own loop.

_loop = None

def get_loop():
    global _loop

    if _loop is None: # could try to re-use the existing one if you want
         _loop = asyncio.get_new_loop()
    return _loop

A workaround for anyone getting this on the main CLI is just use --watch which wraps everything in asyncio.run and "fixes" it.

arq path.to.Queue --watch /dev/null