Wheel installer assumes download path

[matzipan]

🐞 bug report

Affected Rule

pip_parse

Is this a regression?

Unsure

Description

The wheel installer glob assumes that the wheels will be downloded to the current directory. The call to pip is done without -w, and this does not guarantee the download path.

For example, when /etc/pip.conf has the following lines:

[wheel]
wheel-dir = /var/cache/pip

The wheel installer will throw an exception (see below).

Adding -w . fixes the issue and allows the glob to find the wheels.

The code would look like:

    pip_args = (
        [sys.executable, "-m", "pip"]
        + (["--isolated"] if args.isolated else [])
        + (["download", "--only-binary=:all:"] if args.download_only else ["wheel"])
        + ["--no-deps"]
        + ["-w", "."]
        + deserialized_args["extra_pip_args"]
    )

🔬 Minimal Reproduction

Have /etc/pip.conf with the following lines:

[wheel]
wheel-dir = /var/cache/pip

A pip_parse with a any package from pypi.

It does not matter if you are using the host python or whether you are using hermetic python.

🔥 Exception or Error

INFO: Repository test_framework_deps_iniconfig instantiated at:
  /repo/WORKSPACE:1583:13: in <toplevel>
  /bazel_cache/output_user_root/61da6a75bcd751b8ee01d9c498363d41/external/test_framework_deps/requirements.bzl:77:20: in install_deps
Repository rule whl_library defined at:
  /bazel_cache/output_user_root/61da6a75bcd751b8ee01d9c498363d41/external/rules_python/python/pip_install/pip_repository.bzl:886:30: in <toplevel>
ERROR: An error occurred during the fetch of repository 'test_framework_deps_iniconfig':
   Traceback (most recent call last):
        File "/bazel_cache/output_user_root/61da6a75bcd751b8ee01d9c498363d41/external/rules_python/python/pip_install/pip_repository.bzl", line 723, column 13, in _whl_library_impl
                fail("whl_library %s failed: %s (%s) error code: '%s'" % (rctx.attr.name, result.stdout, result.stderr, result.return_code))
Error in fail: whl_library test_framework_deps_iniconfig failed: Looking in links: /var/cache/pip
Collecting iniconfig==1.1.1 (from -r /tmp/tmppm4b3i38 (line 1))
  File was already downloaded /var/cache/pip/iniconfig-1.1.1-py2.py3-none-any.whl
 (Traceback (most recent call last):
  File "/bazel_cache/output_user_root/61da6a75bcd751b8ee01d9c498363d41/external/python39_x86_64-unknown-linux-gnu/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/bazel_cache/output_user_root/61da6a75bcd751b8ee01d9c498363d41/external/python39_x86_64-unknown-linux-gnu/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/bazel_cache/output_user_root/61da6a75bcd751b8ee01d9c498363d41/external/rules_python/python/pip_install/tools/wheel_installer/wheel_installer.py", line 205, in <module>
    main()
  File "/bazel_cache/output_user_root/61da6a75bcd751b8ee01d9c498363d41/external/rules_python/python/pip_install/tools/wheel_installer/wheel_installer.py", line 198, in main
    whl = Path(next(iter(glob.glob("*.whl"))))
StopIteration
) error code: '1'

🌍 Your Environment

Operating System:

Linux with python in a docker container

Output of bazel version:

Bazel 6.1

Rules_python version:

0.29.0

Anything else relevant?

[aignas]

Please provide logs and a small reproduction example. The fact that our CI works on Windows seems like something is different in your environment.

[matzipan]

I will provide logs but I am not sure I can provide a reprodiction example as I am not sure about how pip chooses its paths. Will try.

[matzipan]

Just to be clear, i am on Linux and not on Windows. I deleted my previous comment since it was not correct.

[matzipan]

Sorry for wasting your time. I should've just waited until I am back in front of the computer to post this report as I have more info. I updated the description and now it's more clear.

It was -w . instead of -t . like I wrote initially, and the cause of this is having wheel_dir configured in /etc/pip.conf.

[matzipan]

After some more trials, I think -w. is the better option instead of -w .. The latter creates a folder called .

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!