pytest icon indicating copy to clipboard operation
pytest copied to clipboard

Test hangs when using mp.Pool on Windows/macOS using CLI runner when --import-mode=importlib

Open DylanLukes opened this issue 2 years ago • 12 comments

This is a duplicate of #5429 (which does not appear to actually be resolved, or has regressed) but I come bearing more details, in particular a matrix that should help diagnose the source of the problem.

Here's a minimal example:

from multiprocessing import Pool

def times_two(x):
    return x * 2

def test_foo():
    with Pool(1) as pool:
        assert pool.map(times_two, [1, 2, 3, 4]) == [2, 4, 6, 8]

This is using pytest 7.3.1.

Tests were performed on Github Actions ubuntu-latest, windows-latest, and macos-latest, which correspond to Ubuntu 22.04, Windows Server 2022, and macOS 12 Monterey (not Ventura). I also validated the macOS and Linux results on macOS 13 Ventura and Ubuntu 20.04 on my own machines.

All of the below were tested with Python 3.11 on Github Actions runners. On my own machines, I tested with both 3.10 and 3.11, same results.

Platform Runner Result
Windows pytest Hangs
macOS pytest Hangs
Linux pytest Completes
macOS PyCharm Completes

The last row in particular suggests that the issue is likely in the test runner. Running all tests via PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py does not hang.

– Dylan

DylanLukes avatar May 06 '23 01:05 DylanLukes