flit
flit copied to clipboard
Published
20 hours ago •
pypa
pypa
Improve isolation of tests for `flit install`
I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
python3 -sBm build -w --no-isolation- because I'm calling
buildwith--no-isolationI'm using during all processes only locally installed modules - install .whl file in </install/prefix>
- run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-flit-3.8.0-6.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-flit-3.8.0-6.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.15, pytest-7.1.3, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/flit-3.8.0
collected 205 items
flit_core/flit_core/tests/test_build_thyself.py ... [ 1%]
flit_core/flit_core/tests/test_buildapi.py ......... [ 5%]
flit_core/flit_core/tests/test_common.py .................... [ 15%]
flit_core/flit_core/tests/test_config.py ............................................. [ 37%]
flit_core/flit_core/tests/test_sdist.py ...... [ 40%]
flit_core/flit_core/tests/test_versionno.py . [ 40%]
flit_core/flit_core/tests/test_wheel.py ..... [ 43%]
tests/test_build.py ..... [ 45%]
tests/test_command.py .. [ 46%]
tests/test_config.py . [ 47%]
tests/test_find_python_executable.py ....... [ 50%]
tests/test_init.py .................. [ 59%]
tests/test_install.py ...F..F..........F.F........ [ 73%]
tests/test_sdist.py ............ [ 79%]
tests/test_tomlify.py . [ 79%]
tests/test_upload.py ....... [ 82%]
tests/test_validate.py ................ [ 90%]
tests/test_vcs.py . [ 91%]
tests/test_wheel.py .................. [100%]
================================================================================= FAILURES =================================================================================
____________________________________________________________________ InstallTests.test_install_data_dir ____________________________________________________________________
self = <tests.test_install.InstallTests testMethod=test_install_data_dir>
def test_install_data_dir(self):
> Installer.from_ini_path(
core_samples_dir / 'with_data_dir' / 'pyproject.toml',
).install_directly()
tests/test_install.py:296:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
flit/install.py:310: in install_directly
self.install_requirements()
flit/install.py:263: in install_requirements
check_call(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1
cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...]
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmp7kwihz9srequirements.txt']' returned non-zero exit status 1.
/usr/lib64/python3.8/subprocess.py:364: CalledProcessError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmp7kwihz9srequirements.txt (line 1)) (2.28.1)
Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmp7kwihz9srequirements.txt (line 2)) (0.18.1)
Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp7kwihz9srequirements.txt (line 1)) (3.4)
Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp7kwihz9srequirements.txt (line 1)) (3.0.0)
Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp7kwihz9srequirements.txt (line 1)) (1.26.12)
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages'
Check the permissions.
_________________________________________________________________ InstallTests.test_install_module_pep621 __________________________________________________________________
self = <tests.test_install.InstallTests testMethod=test_install_module_pep621>
def test_install_module_pep621(self):
> Installer.from_ini_path(
core_samples_dir / 'pep621_nodynamic' / 'pyproject.toml',
).install_directly()
tests/test_install.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
flit/install.py:310: in install_directly
self.install_requirements()
flit/install.py:263: in install_requirements
check_call(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1
cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...]
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmpfoibwf0erequirements.txt']' returned non-zero exit status 1.
/usr/lib64/python3.8/subprocess.py:364: CalledProcessError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmpfoibwf0erequirements.txt (line 1)) (2.28.1)
Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmpfoibwf0erequirements.txt (line 2)) (0.18.1)
Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpfoibwf0erequirements.txt (line 1)) (3.4)
Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpfoibwf0erequirements.txt (line 1)) (1.26.12)
Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpfoibwf0erequirements.txt (line 1)) (3.0.0)
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages'
Check the permissions.
---------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------
WARNING flit_core.versionno:versionno.py:124 Version number normalised: '0.03' -> '0.3' (see PEP 440)
____________________________________________________________________ InstallTests.test_symlink_data_dir ____________________________________________________________________
self = <tests.test_install.InstallTests testMethod=test_symlink_data_dir>
def test_symlink_data_dir(self):
if os.name == 'nt':
raise SkipTest("symlink")
> Installer.from_ini_path(
core_samples_dir / 'with_data_dir' / 'pyproject.toml', symlink=True
).install_directly()
tests/test_install.py:305:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
flit/install.py:310: in install_directly
self.install_requirements()
flit/install.py:263: in install_requirements
check_call(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1
cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...]
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmp1nztui2qrequirements.txt']' returned non-zero exit status 1.
/usr/lib64/python3.8/subprocess.py:364: CalledProcessError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmp1nztui2qrequirements.txt (line 1)) (2.28.1)
Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmp1nztui2qrequirements.txt (line 2)) (0.18.1)
Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp1nztui2qrequirements.txt (line 1)) (3.4)
Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp1nztui2qrequirements.txt (line 1)) (1.26.12)
Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp1nztui2qrequirements.txt (line 1)) (3.0.0)
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages'
Check the permissions.
_________________________________________________________________ InstallTests.test_symlink_module_pep621 __________________________________________________________________
self = <tests.test_install.InstallTests testMethod=test_symlink_module_pep621>
def test_symlink_module_pep621(self):
if os.name == 'nt':
raise SkipTest("symlink")
> Installer.from_ini_path(
core_samples_dir / 'pep621_nodynamic' / 'pyproject.toml', symlink=True
).install_directly()
tests/test_install.py:160:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
flit/install.py:310: in install_directly
self.install_requirements()
flit/install.py:263: in install_requirements
check_call(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1
cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...]
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmpkabqdk4prequirements.txt']' returned non-zero exit status 1.
/usr/lib64/python3.8/subprocess.py:364: CalledProcessError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmpkabqdk4prequirements.txt (line 1)) (2.28.1)
Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmpkabqdk4prequirements.txt (line 2)) (0.18.1)
Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpkabqdk4prequirements.txt (line 1)) (3.4)
Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpkabqdk4prequirements.txt (line 1)) (3.0.0)
Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpkabqdk4prequirements.txt (line 1)) (1.26.12)
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages'
Check the permissions.
---------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------
WARNING flit_core.versionno:versionno.py:124 Version number normalised: '0.03' -> '0.3' (see PEP 440)
========================================================================= short test summary info ==========================================================================
FAILED tests/test_install.py::InstallTests::test_install_data_dir - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...
FAILED tests/test_install.py::InstallTests::test_install_module_pep621 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '...
FAILED tests/test_install.py::InstallTests::test_symlink_data_dir - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...
FAILED tests/test_install.py::InstallTests::test_symlink_module_pep621 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '...
====================================================================== 4 failed, 201 passed in 6.96s =======================================================================
Here is list of installed modules in build env
Package Version
----------------------------- -----------------
alabaster 0.7.12
appdirs 1.4.4
attrs 22.1.0
Babel 2.11.0
Brlapi 0.8.3
build 0.9.0
charset-normalizer 3.0.0
codespell 2.2.1
contourpy 1.0.5
cssselect 1.1.0
cycler 0.11.0
distro 1.7.0
dnspython 2.2.1
docutils 0.18.1
extras 1.0.0
fixtures 4.0.0
fonttools 4.38.0
gpg 1.17.1-unknown
idna 3.4
imagesize 1.4.1
importlib-metadata 5.0.0
iniconfig 1.1.1
Jinja2 3.1.1
kiwisolver 1.4.4
libcomps 0.1.19
louis 3.23.0
lxml 4.9.1
MarkupSafe 2.1.1
matplotlib 3.6.0
numpy 1.23.1
olefile 0.46
packaging 21.3
pbr 5.9.0
pep517 0.13.0
Pillow 9.2.0
pip 22.2.2
pluggy 1.0.0
py 1.11.0
Pygments 2.13.0
PyGObject 3.42.2
pyparsing 3.0.9
pytest 7.1.3
python-dateutil 2.8.2
pytz 2022.4
requests 2.28.1
requests_download 0.1.2
responses 0.22.0
rpm 4.17.0
scour 0.38.2
six 1.16.0
snowballstemmer 2.2.0
Sphinx 5.3.0
sphinx-rtd-theme 1.1.0
sphinxcontrib-applehelp 1.0.2.dev20220730
sphinxcontrib-devhelp 1.0.2.dev20220730
sphinxcontrib-htmlhelp 2.0.0
sphinxcontrib-jsmath 1.0.1.dev20220730
sphinxcontrib-qthelp 1.0.3.dev20220730
sphinxcontrib-serializinghtml 1.1.5
testpath 0.6.0
testtools 2.5.0
toml 0.10.2
tomli 2.0.1
tomli_w 1.0.0
types-toml 0.10.8
urllib3 1.26.12
wheel 0.37.1
zipp 3.9.0
The tests are all failing with
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages'
Check the permissions.
This seems to be a problem with your system and not a problem with the tests.
The test filename is:
tests/test_install.py
That should provide a hint for why its trying to install something. Flit has a flit install command, which is what is being tested in those tests.
Actually, I think there is a genuine issue with the tests here that we hadn't spotted.
The tests do some monkeypatching to redirect installation to temporary directories:
https://github.com/pypa/flit/blob/f5704ea31f0fcc579b8518ea85d641651cba4f71/tests/test_install.py#L25-L30
But this doesn't affect the _auto_user method, which looks to see if it should do a user install or not:
https://github.com/pypa/flit/blob/f5704ea31f0fcc579b8518ea85d641651cba4f71/flit/install.py#L152-L154
And of course the monkeypatching also won't affect pip when the install code runs that as a subprocess.
So I'm going to reopen this :wink: . I'm not sure yet exactly what's the best way to properly isolate those tests - maybe we have to set up a venv for each test, even if that's slower - but I think they do need some work.
