Windows 3.7 or 3.8 multiprocessing failure on 20.0.34, not with 20.0.33

[nedbat]

Issue

The coverage.py test suite fails on Appveyor with virtualenv 20.0.34: one particular multiprocessing test gets stuck and is eventually killed. This happens with Python 3.7 or 3.8.

Moving back to virtualenv 20.0.33 makes the problem go away.

Here are Appveyor runs:

• with 20.0.33 (succeeds): https://ci.appveyor.com/project/nedbat/coveragepy/builds/35850491
• with 20.0.34 (gets stuck and killed): https://ci.appveyor.com/project/nedbat/coveragepy/builds/35848404

I don't have Windows easily at hand, so I haven't reproduced it myself, but here is how you should be able to:

# in a Windows Python 3.8 virtualenv..
git clone https://github.com/nedbat/coveragepy
cd coveragepy
git checkout nedbat/virtualenv-34
pip install tox
tox -e py38

The test that gets stuck is "test_bug890", you can run it in isolation with:

tox -e py38 -- -k bug890

To try the same code on virtualenv 20.0.33, use:

git checkout nedbat/virtualenv-33

[gaborbernat]

On windows we now align with how venv works, so this likely will be a venv Cpython bug 🤷‍♂️ without a better reproducible will be hard to find out... 🤔

[nedbat]

What else can I give you to reproduce it? Did the steps I specified not show the problem?

[gaborbernat]

Can you show what the test does exactly?

[MarkKoz]

I cannot reproduce this locally on 3.7.9 on Windows.

[nedbat]

This is the test:

    def test_bug890(self):
        # chdir in multiprocessing shouldn't keep us from finding the
        # .coveragerc file.
        self.make_file("multi.py", """\
            import multiprocessing, os, os.path
            if __name__ == "__main__":
                if not os.path.exists("./tmp"): os.mkdir("./tmp")
                os.chdir("./tmp")
                with multiprocessing.Manager():
                    pass
                print("ok")
            """)
        self.make_file(".coveragerc", """\
            [run]
            concurrency = multiprocessing
            """)
        out = self.run_command("coverage run multi.py")
        self.assertEqual(out.splitlines()[-1], "ok")

I know this isn't as helpful as a more focused reproducer. I don't have a way to reduce the code (because I don't have Windows).

[gaborbernat]

• Eh when you call subprocess are you doing it via fully qualified path? You could be experiencing a variation of https://bugs.python.org/issue42041

[gaborbernat]

So you're passing on shell=True per https://github.com/nedbat/coveragepy/blob/master/tests/helpers.py#L37:

With shell=True, the CMD shell simply checks %CD% (unless %NoDefaultCurrentDirectoryInExePath% is defined and the name to search contains no slashes) and %PATH%. Support for forward slash in the name to search is wonky; it works only for quoted paths. But at least a relative path that contains slashes is only resolved against the current working directory instead of every directory in the search path. CMD's rules for appending default extensions are simpler than SearchPathW in some ways, but also more complicated because it's generalized as the PATHEXT list of extensions. In each directory, CMD always looks first for the searched name as given and then the name plus each extension in PATHEXT, regardless of the filepath type or whether the searched name already has an extension. It will not find a name that has no extension unless PATHEXT includes a "." entry for the empty extension. (This is consistent with the desktop shell API, which supports defining an association for the "." filetype.)

It makes me think maybe it's not related to that issue in particular.

I don't have Windows easily at hand,

Takes a while to download, but you can get one, for free, via https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/. Pick your favorite virtualization machine type, and away you go.

[gaborbernat]

Guess this is no longer the case as no one reported it.