pytest-dev
8.0.0 + pyargs + permissions causes failures
Example of a successful test-install build on 7.4.4:
https://github.com/pulkin/pyteleport/actions/runs/7603885071/job/20706143053
On 8.0.0 it starts looking into inaccessible folders and fails right away:
https://github.com/pulkin/pyteleport/actions/runs/7733355863/job/21085241736
This has been reported a few times already. The previous cases were legitimately bad ideas, running tests from /tmp. But since it seems like a common issue, we probably should make it work.
The easiest thing is to simply ignore files/dirs which raise a permission error, however I don't like this because it may silently skip over files which are meant to be included and really have wrong perms.
Another easy option is to warn and ignore, but the people who report this probably don't want the noise.
The problem started happening in pytest 8 because of the Session collection changes. Previously Session would collect the initial files itself, and thus could filter out non-matching files early. Now Session only collects the initial Directorys and then does a "matchnodes" pass after the fact, which means some files now get stat'ed when they previously weren't.
With some effort we could probably go back to avoiding stats on files that are going to get discarded anyway. However it is probably too big a change for 8.0.x so will only be fixed in a feature release.
I am all ears if you come up with a solution to test an installed package in my case. I used cd / to distance from my src folder as far as it is possible before running pytest.
I do recommend cd'ing into some empty directory your user owns, or at least /var/empty if you need something ready made.
To whomever interested, I did this
https://github.com/pulkin/pyteleport/commit/8a5e87c3956114f26a417bbe48bd28910bd46e7c
Creating an empty subfolder in the repo folder did not work because pytest can still find setup.py and use repo files. So you have to create an empty folder side-by-side and test from there.
I too am experiencing similar issues, am curious about a way forward. Given my use case, I do not have the option of creating a directory and cd'ing into it given my working environment.
This is an issue for my org.
We deploy applications fully isolated, so they all ship with their own python deps, similar to venv.
We have a generic test tool that allows running pytest via --pyargs, however the collector now doesn't stay within the confines of the package and starts adding dirs starting from root which contains tons of files and folders that it isn't permissioned to look at.
The typical invocation would look something like:
/root/packages/libexec/mypackage/bin/python -m pytest --pyargs myorg.myteam.mypackage
With v8, looking at pytest debug file, this starts traversing /root first, encountering all sorts of things. Instead, what we expect is that it will stay within the confines of where the module is installed, i.e.
/root/packages/libexec/mypackage/python/lib/python3.11/site-packages/myorg/myteam/mypackage
As a workaround, I can simply run pytest on the folder ...myorg/myteam/mypackage but this is far less elegant than --pyargs that should just resolve this.
Another possible workaround is specifying --confcutdir to the directory you want pytest to start to and below.
But I agree the current situation is not optimal.
I'm seeing something which looks like this issue in line-profiler.
The PR containing the issue is: https://github.com/pyutils/line_profiler/pull/257
When I pin the pytest version to pytest<=7.4.4, everything works correctly. This run shows that behavior: https://github.com/pyutils/line_profiler/actions/runs/8863760338
But if I unpin pytest I start getting permission errors in osx and win32 builds inside the cibuildwheel job. https://github.com/pyutils/line_profiler/actions/runs/8863904478
The invocation of pytest is inside another Python file: run_tests.py (link is to the version in the PR) I'm doing import pytest and then pytest.main(pytest_args). Is there anything I can do in the surrounding script to ensure pytest doesn't try to check these directories?
Errors I get look like:
=================================== ERRORS ====================================
________________________ ERROR collecting test session ________________________
C:\Users\runneradmin\AppData\Local\Temp\cibw-run-muf21jxz\cp38-win_amd64\venv-test\lib\site-packages\_pytest\runner.py:341: in from_call
result: Optional[TResult] = func()
C:\Users\runneradmin\AppData\Local\Temp\cibw-run-muf21jxz\cp38-win_amd64\venv-test\lib\site-packages\_pytest\runner.py:389: in collect
return list(collector.collect())
C:\Users\runneradmin\AppData\Local\Temp\cibw-run-muf21jxz\cp38-win_amd64\venv-test\lib\site-packages\_pytest\main.py:523: in collect
for direntry in scandir(self.path):
C:\Users\runneradmin\AppData\Local\Temp\cibw-run-muf21jxz\cp38-win_amd64\venv-test\lib\site-packages\_pytest\pathlib.py:896: in scandir
with os.scandir(path) as s:
E PermissionError: [WinError 5] Access is denied: 'C:\\Documents and Settings'
---------- coverage: platform win32, python 3.8.10-final-0 -----------
...
=========================== short test summary info ===========================
ERROR ::::Documents and Settings - PermissionError: [WinError 5] Access is denied: 'C:\\Documents and Settings'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 1.39s ===============================
and on OSX:
============================= test session starts ==============================
platform darwin -- Python 3.8.10, pytest-8.2.0, pluggy-1.5.0
rootdir: /Users/runner/work/line_profiler/line_profiler
configfile: pyproject.toml
plugins: cov-5.0.0, xdoctest-1.1.3
collected 0 items / 1 error
==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/cibw-run-cy6pklkf/cp38-macosx_x86_64/venv-test-x86_64/lib/python3.8/site-packages/pluggy/_hooks.py:513: in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/cibw-run-cy6pklkf/cp38-macosx_x86_64/venv-test-x86_64/lib/python3.8/site-packages/pluggy/_manager.py:120: in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/cibw-run-cy6pklkf/cp38-macosx_x86_64/venv-test-x86_64/lib/python3.8/site-packages/_pytest/python.py:179: in pytest_collect_directory
if pkginit.is_file():
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pathlib.py:1439: in is_file
return S_ISREG(self.stat().st_mode)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pathlib.py:1198: in stat
return self._accessor.stat(self)
E PermissionError: [Errno 13] Permission denied: '/private/var/audit/__init__.py'
=========================== short test summary info ============================
ERROR ../::private::var - PermissionError: [Errno 13] Permission denied: '/private/var/audit/__init__.py'
=============================== 1 error in 0.16s ===============================
[run_tests] pytest returned ret=4
[run_tests] Restoring cwd = '/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/cibw-run-cy6pklkf/cp38-macosx_x86_64/test_cwd'
