hatch icon indicating copy to clipboard operation
hatch copied to clipboard

As of 1.16, tests no longer work offline (they try to install packages from the Internet)

Open mgorny opened this issue 4 weeks ago • 2 comments

In 1.16.1 release, multiple tests are now failing in network-isolated environments while trying to install Python packages. They are not marked as requiring network. I can reproduce with 843f91680cc99b7e68e407a5afd7e98a8eb65ca1 too.

Example reproducer:

$ uv venv -p 3.12
Using CPython 3.12.12 interpreter at: /usr/bin/python3.12
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
$ uv pip install -e . editables flit-core filelock trustme pyfakefs pytest-mock pytest-xdist
Resolved 51 packages in 1.60s
      Built hatch @ file:///tmp/hatch
Prepared 45 packages in 651ms
Installed 51 packages in 32ms
 + anyio==4.12.0
 + backports-zstd==1.1.0
 + certifi==2025.11.12
 + cffi==2.0.0
 + click==8.3.1
 + cryptography==46.0.3
 + distlib==0.4.0
 + editables==0.5
 + execnet==2.1.2
 + filelock==3.20.0
 + flit-core==3.12.0
 + h11==0.16.0
 + hatch==1.16.2.dev2 (from file:///tmp/hatch)
 + hatchling==1.28.0
 + httpcore==1.0.9
 + httpx==0.28.1
 + hyperlink==21.0.0
 + idna==3.11
 + iniconfig==2.3.0
 + jaraco-classes==3.4.0
 + jaraco-context==6.0.1
 + jaraco-functools==4.3.0
 + jeepney==0.9.0
 + keyring==25.7.0
 + markdown-it-py==4.0.0
 + mdurl==0.1.2
 + more-itertools==10.8.0
 + packaging==25.0
 + pathspec==0.12.1
 + pexpect==4.9.0
 + platformdirs==4.5.0
 + pluggy==1.6.0
 + ptyprocess==0.7.0
 + pycparser==2.23
 + pyfakefs==5.10.2
 + pygments==2.19.2
 + pyproject-hooks==1.2.0
 + pytest==9.0.1
 + pytest-mock==3.15.1
 + pytest-xdist==3.8.0
 + rich==14.2.0
 + secretstorage==3.5.0
 + shellingham==1.5.4
 + tomli-w==1.2.0
 + tomlkit==0.13.3
 + trove-classifiers==2025.11.14.15
 + trustme==1.2.1
 + typing-extensions==4.15.0
 + userpath==1.9.2
 + uv==0.9.13
 + virtualenv==20.35.4
$ rm -r ~/.cache/uv
$ sudo unshare -n su $USER
$ .venv/bin/pytest -n12 -m "not requires_internet and not requires_docker"
========================================================= test session starts =========================================================
platform linux -- Python 3.12.12, pytest-9.0.1, pluggy-1.6.0
rootdir: /tmp/hatch
configfile: pyproject.toml
plugins: pyfakefs-5.10.2, anyio-4.12.0, xdist-3.8.0, mock-3.15.1
12 workers [2123 items]   
..........s.........s..............s.s...............................................s..s...s....................s............. [  5%]
....................s...s....s.s.s.s............................................................s.............................. [ 11%]
..............s.s..................s......................................s.................................................... [ 17%]
............................................F.................................................................................. [ 23%]
............................................................................................................................... [ 29%]
............................................................................................................................... [ 35%]
............................................................................................................................... [ 41%]
............................................................................................................s.................. [ 47%]
...................sssss..................................s.................................................................... [ 53%]
............................................................................................................................... [ 59%]
............................................................................................................................... [ 65%]
............................................................................................................................... [ 71%]
............................................................................................................................... [ 77%]
............................................................................................................................... [ 83%]
..............................................................................................................sssssssssssss.sss [ 89%]
.........................................................F............................F.................FFFFF.FFF.F.FFFFFFFFFFF [ 95%]
FFFFFFFFFFFFFFFFF...................FF.....................FF.FFFF.........FFF.............                                     [100%]
============================================================== FAILURES ===============================================================
_______________________________________________ TestSBOMFiles.test_sbom_from_build_data _______________________________________________
[gw10] linux -- Python 3.12.12 /tmp/hatch/.venv/bin/python3

self = <tests.backend.builders.test_wheel.TestSBOMFiles object at 0x7f6678450dd0>
hatch = <tests.conftest.CliRunner object at 0x7f6673ecf860>
helpers = <module 'tests.helpers.helpers' from '/tmp/hatch/tests/helpers/helpers.py'>, temp_dir = Path('/tmp/tmprz5ffqq1')
config_file = <hatch.config.user.ConfigFile object at 0x7f6673401010>

    def test_sbom_from_build_data(self, hatch, helpers, temp_dir, config_file):
        config_file.model.template.plugins["default"]["tests"] = False
        config_file.save()
    
        with temp_dir.as_cwd():
            result = hatch("new", "My.App")
    
        assert result.exit_code == 0, result.output
    
        project_path = temp_dir / "my-app"
        (project_path / "sbom1.cyclonedx.json").write_text('{"bomFormat": "CycloneDX"}')
        (project_path / "sbom2.spdx.json").write_text('{"spdxVersion": "SPDX-2.3"}')
    
        build_script = project_path / DEFAULT_BUILD_SCRIPT
        build_script.write_text(
            helpers.dedent(
                """
                import pathlib
    
                from hatchling.builders.hooks.plugin.interface import BuildHookInterface
    
                class CustomHook(BuildHookInterface):
                    def initialize(self, version, build_data):
                        build_data["sbom_files"].append("sbom2.spdx.json")
                """
            )
        )
    
        config = {
            "project": {"name": "My.App", "dynamic": ["version"]},
            "tool": {
                "hatch": {
                    "version": {"path": "src/my_app/__about__.py"},
                    "build": {
                        "targets": {"wheel": {"sbom-files": ["sbom1.cyclonedx.json"]}},
                        "hooks": {"custom": {"path": DEFAULT_BUILD_SCRIPT}},
                    },
                }
            },
        }
        builder = WheelBuilder(str(project_path), config=config)
    
        build_path = project_path / "dist"
        build_path.mkdir()
    
        with project_path.as_cwd():
>           artifacts = list(builder.build(directory=str(build_path)))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/tmp/hatch/tests/backend/builders/test_wheel.py:3972: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/tmp/hatch/.venv/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py:149: in build
    build_hook.initialize(version, build_data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <hatch_build.py.CustomHook object at 0x7f66735ce270>, version = 'standard'
build_data = {'artifacts': [], 'build_hooks': ('custom',), 'dependencies': [], 'extra_metadata': {}, ...}

    def initialize(self, version, build_data):
>       build_data["sbom_files"].append("sbom2.spdx.json")
        ^^^^^^^^^^^^^^^^^^^^^^^^
E       KeyError: 'sbom_files'

/tmp/tmprz5ffqq1/my-app/hatch_build.py:7: KeyError
________________________________________________________ test_unknown_targets _________________________________________________________
[gw10] linux -- Python 3.12.12 /tmp/hatch/.venv/bin/python3

hatch = <tests.conftest.CliRunner object at 0x7f6673ecf860>, temp_dir = Path('/tmp/tmpcdoyly4a')
helpers = <module 'tests.helpers.helpers' from '/tmp/hatch/tests/helpers/helpers.py'>

    def test_unknown_targets(hatch, temp_dir, helpers):
        project_name = "My.App"
    
        with temp_dir.as_cwd():
            result = hatch("new", project_name)
            assert result.exit_code == 0, result.output
    
        path = temp_dir / "my-app"
    
        with path.as_cwd():
            result = hatch("build", "-t", "foo")
    
>       assert result.exit_code == 1, result.output
E       AssertionError: Creating environment: hatch-build
E         Checking dependencies
E         Syncing dependencies
E         
E       assert 2 == 1
E        +  where 2 = <Result SystemExit(2)>.exit_code

/tmp/hatch/tests/cli/build/test_build.py:250: AssertionError
-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
error: Failed to fetch: `https://pypi.org/simple/hatchling/`
  Caused by: Request failed after 3 retries
  Caused by: error sending request for url (https://pypi.org/simple/hatchling/)
  Caused by: client error (Connect)
  Caused by: dns error
  Caused by: failed to lookup address information: Name or service not known
____________________________________________________ test_verbose_output_to_stderr ____________________________________________________
[gw11] linux -- Python 3.12.12 /tmp/hatch/.venv/bin/python3

hatch = <tests.conftest.CliRunner object at 0x7f2793b067b0>, temp_dir = Path('/tmp/tmplbooz4ca')

    @pytest.mark.usefixtures("mock_backend_process")
    def test_verbose_output_to_stderr(hatch, temp_dir):
        """Test that verbose output (command display and status messages) goes to stderr, not stdout."""
        project_name = "My.App"
    
        with temp_dir.as_cwd():
            hatch("new", project_name)
    
        path = temp_dir / "my-app"
        data_path = temp_dir / "data"
        data_path.mkdir()
    
        # Run with verbose flag (-v) and separate stderr from stdout
        with path.as_cwd(env_vars={ConfigEnvVars.DATA: str(data_path)}):
            result = hatch("-v", "version")
    
>       assert result.exit_code == 0, result.output
E       AssertionError: Creating environment: hatch-build
E         Checking dependencies
E         Syncing dependencies
E         
E       assert 2 == 0
E        +  where 2 = <Result SystemExit(2)>.exit_code

/tmp/hatch/tests/cli/version/test_version.py:388: AssertionError
-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
Using CPython 3.12.12 interpreter at: /tmp/hatch/.venv/bin/python3
Creating virtual environment at: /tmp/tmplbooz4ca/data/env/virtual/my-app/ODC-L8f_/hatch-build
Using Python 3.12.12 environment at: /tmp/tmplbooz4ca/data/env/virtual/my-app/ODC-L8f_/hatch-build
error: Failed to fetch: `https://pypi.org/simple/hatchling/`
  Caused by: Request failed after 3 retries
  Caused by: error sending request for url (https://pypi.org/simple/hatchling/)
  Caused by: client error (Connect)
  Caused by: dns error
  Caused by: failed to lookup address information: Name or service not known
[…]
======================================================= short test summary info =======================================================
FAILED tests/backend/builders/test_wheel.py::TestSBOMFiles::test_sbom_from_build_data - KeyError: 'sbom_files'
FAILED tests/cli/build/test_build.py::test_unknown_targets - AssertionError: Creating environment: hatch-build
FAILED tests/cli/version/test_version.py::test_verbose_output_to_stderr - AssertionError: Creating environment: hatch-build
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_build_all_members - assert 2 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_member_features - assert 1 == 0
FAILED tests/cli/build/test_build.py::test_clean_only_hooks_only - AssertionError: Creating environment: hatch-build
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_development_workflow - assert 2 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_parallel_dependency_resolution - assert 1 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_exclude_patterns - assert 1 == 0
FAILED tests/cli/build/test_build.py::test_mutually_exclusive_hook_options - AssertionError: Creating environment: hatch-build
FAILED tests/cli/version/test_version.py::test_other_backend_show - AssertionError: Creating environment: hatch-build
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_library_with_plugins - assert 2 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_members_editable_install - assert 1 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_overrides_platform_conditional_members - assert 1 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_overrides_matrix_conditional_members - assert 1 == 0
FAILED tests/cli/build/test_build.py::test_clean_hooks_after - AssertionError: Creating environment: hatch-build
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_multi_service_application - assert 2 == 0
FAILED tests/cli/new/test_new.py::test_initialize_setup_cfg_only - AssertionError: Migrating project metadata from setuptools
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_environment_specific_workspace_slices - assert 1 == 0
FAILED tests/cli/build/test_build.py::test_default - AssertionError: Creating environment: hatch-build
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_cross_member_dependencies - assert 1 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_documentation_generation - assert 1 == 0
FAILED tests/cli/build/test_build.py::test_clean_hooks_after_env_var - AssertionError: Creating environment: hatch-build
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_overrides_combined_conditions - assert 1 == 0
FAILED tests/workspaces/test_config.py::TestWorkspaceConfiguration::test_workspace_no_members_fallback - assert 1 == 0
FAILED tests/cli/version/test_version.py::test_show_dynamic - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_explicit_targets - AssertionError: Creating environment: hatch-build
FAILED tests/cli/project/test_metadata.py::test_other_backend - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_explicit_directory - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_clean_only_no_hooks - AssertionError: Creating environment: hatch-build
FAILED tests/cli/version/test_version.py::test_plugin_dependencies_unmet - AssertionError: Syncing environment plugin requirements
FAILED tests/cli/project/test_metadata.py::test_default_all - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_hooks_only - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_explicit_directory_env_var - AssertionError: Creating environment: hatch-build
FAILED tests/cli/version/test_version.py::test_set_dynamic - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_clean - AssertionError: Creating environment: hatch-build
FAILED tests/cli/project/test_metadata.py::test_field_readme - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_hooks_only_env_var - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_clean_env_var - AssertionError: Creating environment: hatch-build
FAILED tests/cli/version/test_version.py::test_set_dynamic_downgrade - AssertionError: Creating environment: hatch-build
FAILED tests/cli/project/test_metadata.py::test_field_string - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_clean_only - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_extensions_only - AssertionError: Creating environment: hatch-build
FAILED tests/cli/project/test_metadata.py::test_field_complex - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_no_hooks - AssertionError: Creating environment: hatch-build
FAILED tests/cli/project/test_metadata.py::test_plugin_dependencies_unmet - AssertionError: Syncing environment plugin requirements
FAILED tests/cli/build/test_build.py::test_no_hooks_env_var - AssertionError: Creating environment: hatch-build
FAILED tests/cli/project/test_metadata.py::test_build_dependencies_unmet - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_debug_verbosity - AssertionError: Creating environment: hatch-build
FAILED tests/cli/build/test_build.py::test_plugin_dependencies_unmet - AssertionError: Syncing environment plugin requirements
FAILED tests/cli/clean/test_clean.py::test - AssertionError: Creating environment: hatch-build
============================================ 51 failed, 2030 passed, 42 skipped in 47.06s =============================================

Note that you may need to remove uv's cache first to reproduce.

Full output (140k): pytest.txt

mgorny avatar Nov 29 '25 11:11 mgorny

I will try to repro this. One note - we no longer use pyfakefs as of 1.16.0 because it was used in 2 places and did not work correctly with coverage on 3.14 until a recent fix was made.

cjames23 avatar Nov 30 '25 04:11 cjames23

@cjames23 I'm also running into this situation. My host is behind a corporate proxy and since v1.16.0 we are unable to install any package using hatch.

Reverting to <1.16.0 fixes the issue.

gaby avatar Dec 04 '25 02:12 gaby