pytest-xvfb
pytest-xvfb copied to clipboard
The-Compiler
Some tests failing on Debian with pytest-xvfb 3.0.0
Hello! I'm trying to upgrade the Debian version of this package to 3.0.0, but some of the tests are now failing. I don't know why this might be, as they run fine on GitHub Actions. Here are the failing tests (the xephyr tests fail in the same way if xserver-xephyr is installed; they are not included here). I wonder if you have any idea what might be wrong and how we might address it? It seems that the --no-xvfb marker might be being ignored? As a comparison, the tests work fine on 2.0.0; if you want to look at my attempt at packaging 3.0.0 for Debian, the git repo is at https://salsa.debian.org/python-team/packages/pytest-xvfb
$ python3.11 -m pytest tests/test_xvfb.py::test_empty_display tests/test_xvfb.py::test_no_xvfb_marker tests/test_xvfb.py::test_xvfb_with_xauth
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /build/pytest-xvfb-eZZpZI/pkg-tests
plugins: xvfb-3.0.0
collected 16 items
tests/test_xvfb.py FFss.F.FssssFFss [100%]
=================================== FAILURES ===================================
___________________________ test_empty_display[None] ___________________________
pytester = <Pytester PosixPath('/tmp/pytest-of-user/pytest-13/test_empty_display0')>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fdff0f27d90>
backend_args = []
def test_empty_display(
pytester: pytest.Pytester, monkeypatch: pytest.MonkeyPatch, backend_args: list[str]
) -> None:
if backend_args == ["--xvfb-backend", "xephyr"]:
pytest.skip("Xephyr needs a host display")
monkeypatch.setenv("DISPLAY", "")
pytester.makepyfile(
"""
import os
def test_display():
assert 'DISPLAY' in os.environ
"""
)
result = pytester.runpytest(*backend_args)
> assert os.environ["DISPLAY"] == ""
E AssertionError: assert ':152' == ''
E + :152
/build/pytest-xvfb-eZZpZI/pkg-tests/tests/test_xvfb.py:84: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/pytest-of-user/pytest-13/test_empty_display0
plugins: xvfb-3.0.0
collected 1 item
test_empty_display.py . [100%]
============================== 1 passed in 0.01s ===============================
___________________________ test_empty_display[xvfb] ___________________________
pytester = <Pytester PosixPath('/tmp/pytest-of-user/pytest-13/test_empty_display1')>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fdff095d690>
backend_args = ['--xvfb-backend', 'xvfb']
def test_empty_display(
pytester: pytest.Pytester, monkeypatch: pytest.MonkeyPatch, backend_args: list[str]
) -> None:
if backend_args == ["--xvfb-backend", "xephyr"]:
pytest.skip("Xephyr needs a host display")
monkeypatch.setenv("DISPLAY", "")
pytester.makepyfile(
"""
import os
def test_display():
assert 'DISPLAY' in os.environ
"""
)
result = pytester.runpytest(*backend_args)
> assert os.environ["DISPLAY"] == ""
E AssertionError: assert ':153' == ''
E + :153
/build/pytest-xvfb-eZZpZI/pkg-tests/tests/test_xvfb.py:84: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/pytest-of-user/pytest-13/test_empty_display1
plugins: xvfb-3.0.0
collected 1 item
test_empty_display.py . [100%]
============================== 1 passed in 0.00s ===============================
___________________ test_no_xvfb_marker[None-args1-2 passed] ___________________
pytester = <Pytester PosixPath('/tmp/pytest-of-user/pytest-13/test_no_xvfb_marker1')>
args = ['--no-xvfb'], outcome = '2 passed', backend_args = []
@pytest.mark.parametrize(
"args, outcome",
[
([], "1 passed, 1 skipped"),
(["--no-xvfb"], "2 passed"),
],
)
def test_no_xvfb_marker(
pytester: pytest.Pytester, args: list[str], outcome: str, backend_args: list[str]
) -> None:
pytester.makepyfile(
"""
import pytest
@pytest.mark.no_xvfb
def test_marked():
pass
def test_unmarked():
pass
"""
)
res = pytester.runpytest(*args, *backend_args)
> res.stdout.fnmatch_lines(f"*= {outcome}*")
E Failed: nomatch: '*= 2 passed*'
E and: '============================= test session starts =============================='
E and: 'platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0'
E and: 'rootdir: /tmp/pytest-of-user/pytest-13/test_no_xvfb_marker1'
E and: 'plugins: xvfb-3.0.0'
E and: 'collected 2 items'
E and: ''
E and: 'test_no_xvfb_marker.py s. [100%]'
E and: ''
E and: '========================= 1 passed, 1 skipped in 0.00s ========================='
E remains unmatched: '*= 2 passed*'
/build/pytest-xvfb-eZZpZI/pkg-tests/tests/test_xvfb.py:243: Failed
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/pytest-of-user/pytest-13/test_no_xvfb_marker1
plugins: xvfb-3.0.0
collected 2 items
test_no_xvfb_marker.py s. [100%]
========================= 1 passed, 1 skipped in 0.00s =========================
___________________ test_no_xvfb_marker[xvfb-args1-2 passed] ___________________
pytester = <Pytester PosixPath('/tmp/pytest-of-user/pytest-13/test_no_xvfb_marker3')>
args = ['--no-xvfb'], outcome = '2 passed'
backend_args = ['--xvfb-backend', 'xvfb']
@pytest.mark.parametrize(
"args, outcome",
[
([], "1 passed, 1 skipped"),
(["--no-xvfb"], "2 passed"),
],
)
def test_no_xvfb_marker(
pytester: pytest.Pytester, args: list[str], outcome: str, backend_args: list[str]
) -> None:
pytester.makepyfile(
"""
import pytest
@pytest.mark.no_xvfb
def test_marked():
pass
def test_unmarked():
pass
"""
)
res = pytester.runpytest(*args, *backend_args)
> res.stdout.fnmatch_lines(f"*= {outcome}*")
E Failed: nomatch: '*= 2 passed*'
E and: '============================= test session starts =============================='
E and: 'platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0'
E and: 'rootdir: /tmp/pytest-of-user/pytest-13/test_no_xvfb_marker3'
E and: 'plugins: xvfb-3.0.0'
E and: 'collected 2 items'
E and: ''
E and: 'test_no_xvfb_marker.py s. [100%]'
E and: ''
E and: '========================= 1 passed, 1 skipped in 0.00s ========================='
E remains unmatched: '*= 2 passed*'
/build/pytest-xvfb-eZZpZI/pkg-tests/tests/test_xvfb.py:243: Failed
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/pytest-of-user/pytest-13/test_no_xvfb_marker3
plugins: xvfb-3.0.0
collected 2 items
test_no_xvfb_marker.py s. [100%]
========================= 1 passed, 1 skipped in 0.00s =========================
__________________________ test_xvfb_with_xauth[None] __________________________
pytester = <Pytester PosixPath('/tmp/pytest-of-user/pytest-13/test_xvfb_with_xauth0')>
backend_args = []
@pytest.mark.skipif(not xauth_available, reason="no xauth")
def test_xvfb_with_xauth(pytester: pytest.Pytester, backend_args: list[str]) -> None:
original_auth = os.environ.get("XAUTHORITY")
pytester.makeini(
"""
[pytest]
xvfb_xauth = True
"""
)
pytester.makepyfile(
"""
import os
def test_xauth():
print('\\nXAUTHORITY: ' + os.environ['XAUTHORITY'])
assert os.path.isfile(os.environ['XAUTHORITY'])
assert os.access(os.environ['XAUTHORITY'], os.R_OK)
"""
)
result = pytester.runpytest("-s", *backend_args)
# Get and parse the XAUTHORITY: line
authline = next(l for l in result.outlines if l.startswith("XAUTHORITY:"))
authfile = authline.split(" ", 1)[1]
assert result.ret == 0
# Make sure the authfile is deleted
> assert not os.path.exists(authfile)
E AssertionError: assert not True
E + where True = <function exists at 0x7fdff1e434c0>('/tmp/PyVirtualDisplay.hsrb32v7.Xauthority')
E + where <function exists at 0x7fdff1e434c0> = <module 'posixpath' (frozen)>.exists
E + where <module 'posixpath' (frozen)> = os.path
/build/pytest-xvfb-eZZpZI/pkg-tests/tests/test_xvfb.py:347: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/pytest-of-user/pytest-13/test_xvfb_with_xauth0
configfile: tox.ini
plugins: xvfb-3.0.0
collected 1 item
test_xvfb_with_xauth.py
XAUTHORITY: /tmp/PyVirtualDisplay.hsrb32v7.Xauthority
.
============================== 1 passed in 0.01s ===============================
__________________________ test_xvfb_with_xauth[xvfb] __________________________
pytester = <Pytester PosixPath('/tmp/pytest-of-user/pytest-13/test_xvfb_with_xauth1')>
backend_args = ['--xvfb-backend', 'xvfb']
@pytest.mark.skipif(not xauth_available, reason="no xauth")
def test_xvfb_with_xauth(pytester: pytest.Pytester, backend_args: list[str]) -> None:
original_auth = os.environ.get("XAUTHORITY")
pytester.makeini(
"""
[pytest]
xvfb_xauth = True
"""
)
pytester.makepyfile(
"""
import os
def test_xauth():
print('\\nXAUTHORITY: ' + os.environ['XAUTHORITY'])
assert os.path.isfile(os.environ['XAUTHORITY'])
assert os.access(os.environ['XAUTHORITY'], os.R_OK)
"""
)
result = pytester.runpytest("-s", *backend_args)
# Get and parse the XAUTHORITY: line
authline = next(l for l in result.outlines if l.startswith("XAUTHORITY:"))
authfile = authline.split(" ", 1)[1]
assert result.ret == 0
# Make sure the authfile is deleted
> assert not os.path.exists(authfile)
E AssertionError: assert not True
E + where True = <function exists at 0x7fdff1e434c0>('/tmp/PyVirtualDisplay.pp7i4tj7.Xauthority')
E + where <function exists at 0x7fdff1e434c0> = <module 'posixpath' (frozen)>.exists
E + where <module 'posixpath' (frozen)> = os.path
/build/pytest-xvfb-eZZpZI/pkg-tests/tests/test_xvfb.py:347: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/pytest-of-user/pytest-13/test_xvfb_with_xauth1
configfile: tox.ini
plugins: xvfb-3.0.0
collected 1 item
test_xvfb_with_xauth.py
XAUTHORITY: /tmp/PyVirtualDisplay.pp7i4tj7.Xauthority
.
============================== 1 passed in 0.01s ===============================
=============================== warnings summary ===============================
tests/test_xvfb.py::test_empty_display[None]
/usr/lib/python3.11/subprocess.py:1127: ResourceWarning: subprocess 5808 is still running
_warn("subprocess %s is still running" % self.pid,
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_xvfb.py::test_empty_display[xvfb]
/usr/lib/python3.11/subprocess.py:1127: ResourceWarning: subprocess 5812 is still running
_warn("subprocess %s is still running" % self.pid,
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_xvfb.py::test_no_xvfb_marker[None-args0-1 passed, 1 skipped]
/usr/lib/python3.11/subprocess.py:1127: ResourceWarning: subprocess 5816 is still running
_warn("subprocess %s is still running" % self.pid,
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_xvfb.py::test_no_xvfb_marker[xvfb-args0-1 passed, 1 skipped]
/usr/lib/python3.11/subprocess.py:1127: ResourceWarning: subprocess 5820 is still running
_warn("subprocess %s is still running" % self.pid,
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_xvfb.py::test_xvfb_with_xauth[None]
/usr/lib/python3.11/subprocess.py:1127: ResourceWarning: subprocess 5824 is still running
_warn("subprocess %s is still running" % self.pid,
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_xvfb.py::test_xvfb_with_xauth[xvfb]
/usr/lib/python3.11/subprocess.py:1127: ResourceWarning: subprocess 5829 is still running
_warn("subprocess %s is still running" % self.pid,
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_xvfb.py::test_empty_display[None] - AssertionError: assert ':152' == ''
FAILED tests/test_xvfb.py::test_empty_display[xvfb] - AssertionError: assert ':153' == ''
FAILED tests/test_xvfb.py::test_no_xvfb_marker[None-args1-2 passed] - Failed: nomatch: '*= 2 passed*'
FAILED tests/test_xvfb.py::test_no_xvfb_marker[xvfb-args1-2 passed] - Failed: nomatch: '*= 2 passed*'
FAILED tests/test_xvfb.py::test_xvfb_with_xauth[None] - AssertionError: assert not True
FAILED tests/test_xvfb.py::test_xvfb_with_xauth[xvfb] - AssertionError: assert not True
============== 6 failed, 2 passed, 8 skipped, 6 warnings in 0.86s ==============
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=11>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=13>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=22>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=25>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=15>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=17>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=21>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=23>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=24>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=27>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=26>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=29>
Oh, I've found the problem. The Debian version used the PyPI version of the package, which does not contain pytest.ini, so the new option included there (--runpytest=subprocess) is not passed on to pytest. I guess this could be fixed by adding pytest.ini to the MANIFEST.in file.