cibuildwheel icon indicating copy to clipboard operation
cibuildwheel copied to clipboard

Pyodide doesn't handle specifiers in pip installs (such as we expand in dependency-groups)

Open henryiii opened this issue 6 months ago • 4 comments

Description

In setting up boost-histogram, I added this:

[dependency-groups]
test = [
    "hypothesis>=6.0; sys_platform != 'ios'", # slow on iOS, can fail health check
    "pytest-xdist; sys_platform != 'ios'",  # No processes on iOS
]

this breaks the custom handling in pyodide:

  subprocess.CalledProcessError: Command '['/tmp/cibw-run-rbgbzz8d/cp312-pyodide_wasm32/venv-test/bin/pip', 'install', 'cloudpickle', 'hypothesis>=6.0; sys_platform != "ios"', 'pytest-benchmark', 'pytest>=6.0', 'pytest-xdist; sys_platform != "ios"', 'numpy']' returned non-zero exit status 1.

These should be processed by packaging in pyodide's pip, I think. CC @hoodmane and @agriyakhetarpal.

Build log

https://github.com/scikit-hep/boost-histogram/actions/runs/15434153116/job/43437235111?pr=1018

CI config

https://github.com/scikit-hep/boost-histogram/pull/1018/commits/ffc7dea4715629eec6d4e37592cf28a2af985471

(now in https://github.com/scikit-hep/boost-histogram/pull/1016)

henryiii avatar Jun 04 '25 05:06 henryiii

I think this may be a recent regression caused as part of the fix for the UV double virtual environment bug. I think I fixed it on the main branch of pyodide-build because it broke in the Pyodide test suite. I'll double check and if so I can make a release of pyodide-build and hopefully that'll fix it.

hoodmane avatar Jun 04 '25 16:06 hoodmane

Okay I confirmed that I can reproduce this on pyodide-build 0.30.4 but not on the tip of tree. So I'll release pyodide-build 0.30.5 and that should fix it.

hoodmane avatar Jun 04 '25 17:06 hoodmane

Still seems to be broken:

https://github.com/scikit-hep/boost-histogram/pull/1020 (edit: I'm going to manually work around it in that PR, failing log at https://github.com/scikit-hep/boost-histogram/actions/runs/15483452068/job/43593373177?pr=1020)

  + pip install cloudpickle 'hypothesis>=6.0; sys_platform != "ios"' pytest-benchmark 'pytest>=6.0' 'pytest-xdist; sys_platform != "ios"' numpy
  ERROR: Invalid requirement: '!=': Expected package name at the start of dependency specifier

henryiii avatar Jun 06 '25 05:06 henryiii

We might be able to side step this by using --group.

henryiii avatar Jun 06 '25 15:06 henryiii

I am failing to reproduce this issue in a fresh pyodide venv via pyodide-build==0.30.7 – Pyodide's pip is able to process specifiers just fine. Perhaps this can be closed?

agriyakhetarpal avatar Aug 19 '25 14:08 agriyakhetarpal

I'll see if I can retry this without the workaround after releasing a patch release.

henryiii avatar Aug 19 '25 15:08 henryiii

https://github.com/scikit-hep/boost-histogram/pull/1049 seems to be passing!

henryiii avatar Aug 19 '25 19:08 henryiii