pip wheel fails: No module named 'wheel'

[ClementPinard]

Description

This behaviour happens with pip version 24, and not before.

This happened to me with the package fiftyone-db, but I suppose it would happen with any package that does not have a pyproject.toml for the build requires, and that features a custom wheel class in the setup.py

Here is the related project : https://github.com/voxel51/fiftyone/tree/develop/package/db

As you can see, the code in the setup.py features the following line:

from wheel.bdist_wheel import bdist_wheel

As such, the wheel package is needed at build time, and when I call this command :

pip wheel --no-cache-dir --use-pep517 fiftyone-db

I get the following stackTrace

Collecting fiftyone-db
  Downloading fiftyone_db-1.1.1.tar.gz (7.8 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/home/clementpinard/miniconda3/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/clementpinard/miniconda3/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/clementpinard/miniconda3/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jspla1yj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jspla1yj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-jspla1yj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-jspla1yj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 20, in <module>
      ModuleNotFoundError: No module named 'wheel'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

I am not the owner of this package, but my own package depends on it, which makes the install fail when no binary of this package is available.

I did some tests, and just adding a pyproject.toml with the default recommanded values by PEP518 at the root of the linked folder does the trick.

[build-system]
# Minimum requirements for the build system to execute.
requires = ["setuptools", "wheel"]  # PEP 508 specifications.

As such, I do believe it's related with this PR : #12449 which does not inject wheel dependecy anymore.

My question is then, is this behaviour intended ? If so, I suppose it's not good practice to importe wheel in the setup.py

I am considering making a PR to the fiftyone repo to add the small pyproject.toml file, is there something better to do, that does solve the problem and follow guidelines ?

Thanks for your help ! In the mean time I am stuck with pip 23, which isn't too bad.

Expected behavior

With pip==23.3.2, I get the following message, indicating a successful build:

Collecting fiftyone-db
  Downloading fiftyone_db-1.1.1.tar.gz (7.8 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: fiftyone-db
  Building wheel for fiftyone-db (pyproject.toml) ... done
  Created wheel for fiftyone-db: filename=fiftyone_db-1.1.1-py3-none-manylinux1_x86_64.whl size=42156167 sha256=e10668dca8fe764e423d826896eaad638e609f61f414a9b0e428880d6968f611
  Stored in directory: /tmp/pip-ephem-wheel-cache-7dce0e_e/wheels/1d/e9/44/f939b5ff51803c2b74f42df226db7b40f829158bbc190b8abf
Successfully built fiftyone-db

[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip

pip version

24

Python version

3.11

OS

Ubuntu 22.04

How to Reproduce

pip wheel --no-cache-dir --use-pep517 fiftyone-db

Output

No response

Code of Conduct

• [X] I agree to follow the PSF Code of Conduct.

[chrysle]

My suggestion would be to have a separate dependency on wheel for pip wheel for setup.py-based build system interfaces.

[pfmoore]

I think the situation here is that setuptools automatically adds the wheel dependency as part of its implementation of get_requires_for_build_wheel. But that only supports the internal use of the wheel package by setuptools. In this case, your setup.py has an explicit dependency on the wheel package, and so should explicitly include wheel in the build-system.requires section of your pyproject.toml.

My suggestion would be to have a separate dependency on wheel for pip wheel for setup.py-based build system interfaces.

If you mean pip special-casing this situation, then I disagree. "Explicitly depend on wheel if you explicitly import it" seems like a perfectly reasonable principle to apply here.

[pfmoore]

My question is then, is this behaviour intended ? If so, I suppose it's not good practice to importe wheel in the setup.py

Yes, it's intended behaviour. It's perfectly acceptable practice to import wheel in setup.py, it's just that you need to explicitly declare the dependency, just like you would with anything else you needed to import in setup.py.

[webknjaz]

My suggestion would be to have a separate dependency on wheel for pip wheel for setup.py-based build system interfaces.

@chrysle like Paul said, if somebody imports wheel unconditionally, that's their own direct dependency to declare, not build backend's. And this means it's also their responsibility. setuptools injects that dependency automatically but only when building wheels is requested. What happens here is that the issue author is going through building sdists first and that doesn't pull in the wheel dependency as it's not needed in general. They could of course make it unconditional — that's a decision of the project being packaged. Still, I've been sending variants of #12449 all around the place because of a historic mistake that setuptools was showing wheel in their docs, in the build-system snippet example and it's just wrong (by default, without a context where it'd be needed).

pip install goes though building sdist and then wheel out of it in most (all?) code paths that require building a wheel, AFAIK. So that the ephemeral sdist build env is being created separately from the one for making sdists, I think.

So no, it shouldn't be special-cased, which is why I offered unspecialcasing it in that linked PR a few months back. The issue author was essentially relying on a bug.

@ClementPinard short of adding wheel to your unconditional build dependencies, you could also add a fallback for the import. That won't hurt sdist builds and the wheel builds (including editable installs) will have the wheel distribution provisioned and available in the corresponding ephemeral build envs:

try:
    from wheel.bdist_wheel import bdist_wheel  # building wheels/editables
except ImportError:
    bdist_wheel = object  # building sdists

[webknjaz]

@abravalheri looks like this is something to call out in the setuptools' docs ^

[ClementPinard]

Thanks for the info, the build/sdist distinction makes sense.

Someone is apparently working on a patch for this package in fiftyone, so we're saved.

I suppose I can close this Issue now, unless you want to discuss about setuptools documentation details ;)

[webknjaz]

        File "/tmp/pip-build-env-jspla1yj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jspla1yj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()

This part of the traceback is something that's rather unobvious — setuptools is trying to evaluate setup.py in order to retrieve the build deps (supposedly from setup_requires?) and augment that with wheel. But since this is happening before wheel is recognized by the build frontend and provisioned in the ephemeral build env, it is failing.

I think what pip could do here is recognizing that there's a dependency loop somewhere in the build backend configuration and printing out an clear and unambiguous message sending the users there (additionally to saying that it is likely not a problem with pip).

[webknjaz]

Someone is apparently working on a patch for this package in fiftyone, so we're saved.

Backref: https://github.com/voxel51/fiftyone/issues/4117

I suppose I can close this Issue now, unless you want to discuss about setuptools documentation details ;)

I'd leave this to the pip maintainers to decide if this needs to remain open, once they read the latest comments. Setuptools would probably need a separate issue on their tracker, I just tagged the maintainer above to bring their attention to this.

[abravalheri]

@abravalheri looks like this is something to call out in the setuptools' docs ^

Hi @webknjaz, thank you very much for the comment. As far as I know, wheel API is private, isn't it?

If anyone would like to make a PR to setuptools docs I can have a look. However, I don't feel comfortable documenting the usage of private APIs from other projects.

---

A minor comment: there is an implicit contract that setuptools.build_meta will call the bdist_wheel command. But the wheel package only just provides the reference implementation for it. There is no guarantee that it is the code from wheel that gets called. (At the end of the day this is how the entry-point decoupling mechanism is supposed to work, and achieve extensibility/customisation).

Maybe someday in the future, setuptools will implement its own bdist_wheel, who knows... If that happens,setuptools will stop automatically declaring build dependencies on wheel.

As previously pointed out, it is the responsibility of the developer writing the import that all the dependencies are declared appropriately.

There are many ways of going around it... conditional imports/try...except blocks are just one of them. Explicitly listing dependencies in build-system.requires is another one.

Each decision has their own strengths and disadvantages.

[webknjaz]

However, I don't feel comfortable documenting the usage of private APIs from other projects.

Maybe, not documenting the use of wheel specifically but rather calling out that setup.py gets evaluated and that may cause problems if it makes use of any unguarded imports while not declaring them as build deps. I think, this would be reasonable since setup.py is pretty much supported as a configuration format for setuptools..

[pfmoore]

Closing this as there's nothing for pip here.