securedrop icon indicating copy to clipboard operation
securedrop copied to clipboard

Admin tests fail locally but pass in CI

Open eloquence opened this issue 4 months ago • 2 comments

Steps to reproduce

  1. Run make -C admin test

Expected behavior

All tests pass

Actual behavior

================================================================================== short test summary info ===================================================================================
FAILED tests/test_securedrop-admin-setup.py::TestSecureDropAdmin::test_install_pip_dependencies_up_to_date - FileNotFoundError: [Errno 2] No such file or directory: '/sd-root/admin/.venv3/bin/python3'
FAILED tests/test_securedrop-admin-setup.py::TestSecureDropAdmin::test_install_pip_dependencies_upgraded - FileNotFoundError: [Errno 2] No such file or directory: '/sd-root/admin/.venv3/bin/python3'
FAILED tests/test_securedrop-admin-setup.py::TestSecureDropAdmin::test_install_pip_dependencies_fail - FileNotFoundError: [Errno 2] No such file or directory: '/sd-root/admin/.venv3/bin/python3'
==================================================================== 3 failed, 76 passed, 1 warning in 161.27s (0:02:41) =====================================================================
ERROR: InvocationError for command /usr/bin/env .tox/py3/bin/coverage run --source=securedrop_admin,bootstrap .tox/py3/bin/pytest -v tests (exited with code 1)

Analysis

AFAICT we're bootstrapping the venv into the dev container and use that in CI, but in local development, we're volume-mounting the repository root and using that as the WORKDIR. I assume the tests will be happy if one creates a .venv3, but that shouldn't be necessary and leads to confusing mismatches between CI vs. local dev results.

eloquence avatar Aug 26 '25 23:08 eloquence

We could just error out in the Makefile if the venv doesn't exist. Because it's able to successfully run most of the tests in the dev container, just not the pip-related ones, the current behavior is a bit confusing.

Alternatively, since we're already baking a venv into the Docker image using bootstrap.py, we could copy that one into the volume mount during local runs if the venv is missing.

eloquence avatar Aug 26 '25 23:08 eloquence

Note that this setup is very bad (c.f. #6897) so making bigger/radical changes is totally fine IMO.

legoktm avatar Aug 27 '25 13:08 legoktm