rules_python icon indicating copy to clipboard operation
rules_python copied to clipboard

`whl_filegroup` fails when no python3 installed on host

Open froody opened this issue 1 year ago • 4 comments

🐞 bug report

Affected Rule

The issue is caused by the rule: whl_filegroup

Is this a regression?

No

Description

The whl_filegroup rule runs a python script, but this script depends on /usr/bin/env python3 which in my case isn't present, even though I've set a default python toolchain.

🔬 Minimal Reproduction

## MODULE.bazel
bazel_dep(name = "rules_python", version = "0.34.0")
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(python_version="3.10", is_default=True, ignore_root_user_error = True)
use_repo(python, "python_3_10")
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
    hub_name = "pip_deps",
    python_version = "3.10",
    requirements_lock = "//:requirements.txt",
)
use_repo(pip, "pip_deps")

## BUILD
load("@rules_python//python:pip.bzl", "whl_filegroup")
whl_filegroup(
    name = "numpy_includes",
    pattern = "numpy/core/include/numpy",
    whl = "@pip_deps//numpy:whl",
)

🔥 Exception or Error





15.35 ERROR: /opt/builder/BUILD:57:14: Extracting numpy/core/include/numpy files from ../rules_python~~pip~pip_deps_310_numpy/numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl failed: (Exit 127): extract_wheel_files failed: error executing PyExtractWheelFiles command (from target //:numpy_includes) bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/rules_python~/python/private/whl_filegroup/extract_wheel_files ... (remaining 3 arguments skipped)
15.35 /usr/bin/env: 'python3': No such file or directory

🌍 Your Environment

Operating System:

  
Ubuntu 22.04
  

Output of bazel version:

  
7.2.0
  

Rules_python version:

  
0.34.0
  

Anything else relevant?

froody avatar Jul 24 '24 11:07 froody

Just a quick note, but I think the recently added exec toolchain should help here. This was added for precompiling stuff, but it should be also applicable here.

aignas avatar Jul 25 '24 14:07 aignas

The exec toolchain shouldn't come into play. The relationship is whl_filegroup has cfg=exec dependency on py_binary //python/private/whl_filegroup:extract_wheel_files.

That py_binary, through toolchain resolution, should end up matching the 3.10 toolchain mentioned in the MODULE.bazel. If it's trying to use /usr/bin/env python3, then it's probably ending up in the autodetecting toolchain somehow.

  • Are you doing a cross-build, perhaps?
  • Can you share the full command line for your build?
  • Can you attach the output when running --toolchain_resolution_debug=.*python:toolchain_type.*? (this will be long, so please attach it as a file instead)

rickeylev avatar Jul 30 '24 02:07 rickeylev

Seeing this issue now. Wish this had a resolution!

pcj avatar Sep 23 '25 20:09 pcj

Adding a tag to show that this is up for grabs if anybody in the community wants to contribute.

aignas avatar Sep 24 '25 02:09 aignas