cmake-python-distributions
cmake-python-distributions copied to clipboard
scikit-build
pip install cmake is failing in linux
tep 8/17 : RUN pip install cmake ---> Running in cfc976370495 Collecting cmake Downloading cmake-3.29.5.tar.gz (30 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Installing backend dependencies: started Installing backend dependencies: finished with status 'error' error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 2
╰─> [53 lines of output]
Collecting ninja>=1.5
Downloading ninja-1.11.1.1-py2.py3-none-musllinux_1_1_x86_64.whl.metadata (5.3 kB)
Collecting cmake
Using cached cmake-3.29.5.tar.gz (30 kB)
ERROR: Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
return func(self, options, args)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 377, in run
requirement_set = resolver.resolve(
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in bool
return bool(self._sequence)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in bool
return any(self)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in
note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × pip subprocess to install backend dependencies did not run successfully. │ exit code: 2 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. The command '/bin/sh -c pip install cmake' returned a non-zero code: 1
This also affects me – I am trying to get cmake (as a build requirement for opencv-python) on arm/v7, i.e. from sdist.
Does this mean that cmake cannot bootstrap itself, i.e. it needs some other/older version of cmake to compile?
I have a similar problem
python -m pip install cmake
Defaulting to user installation because normal site-packages is not writeable Collecting cmake Using cached cmake-3.29.5.1.tar.gz (30 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... error error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 2
╰─> [68 lines of output]
Collecting cmake
Using cached cmake-3.29.5.1.tar.gz (30 kB)
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_internal\cli\base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
^^^^^^^^^^^^^^^
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_internal\cli\req_command.py", line 245, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_internal\commands\install.py", line 377, in run
requirement_set = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_internal\resolution\resolvelib\resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_vendor\resolvelib\resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_vendor\resolvelib\resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_vendor\resolvelib\resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_vendor\resolvelib\structs.py", line 156, in bool
return bool(self._sequence)
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_internal\resolution\resolvelib\found_candidates.py", line 155, in bool
return any(self)
^^^^^^^^^
File "C:\Users\vasv1101.local\lib\python3.11-mingw_x86_64_ucrt\site-packages\pip_internal\resolution\resolvelib\found_candidates.py", line 143, in
note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully. │ exit code: 2 ╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.`
Same error here, Raspberry Pi 3 running Ubuntu Server 20.04.5 LTS (32-bit). Yes this is an unusual configuration, but this hardware and OS seem to be the only working solution for a tool I require. Full error output at the bottom.
This happens to me after creating a new venv using python -m venv <directory> and activating it using source <directory>/bin/activate.
One workaround is to pass
--no-build-isolationtopip install.
Unfortunately for me this fails with scikit_build_core.errors.NinjaNotFoundError: Ninja is required to build
And installing Ninja fails with ModuleNotFoundError: No module named 'cmake'
So it appears I'm out of luck with getting this venv working for the time being.
Full error output:
pip install cmake --verbose
Using pip 24.0 from /home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip (python 3.8)
Collecting cmake
Using cached cmake-3.29.5.1.tar.gz (30 kB)
Running command pip subprocess to install build dependencies
Collecting scikit-build-core
Using cached scikit_build_core-0.9.6-py3-none-any.whl.metadata (19 kB)
Collecting exceptiongroup>=1.0 (from scikit-build-core)
Using cached exceptiongroup-1.2.1-py3-none-any.whl.metadata (6.6 kB)
Collecting importlib-resources>=1.3 (from scikit-build-core)
Using cached importlib_resources-6.4.0-py3-none-any.whl.metadata (3.9 kB)
Collecting packaging>=21.3 (from scikit-build-core)
Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting pathspec>=0.10.1 (from scikit-build-core)
Using cached pathspec-0.12.1-py3-none-any.whl.metadata (21 kB)
Collecting tomli>=1.2.2 (from scikit-build-core)
Using cached tomli-2.0.1-py3-none-any.whl.metadata (8.9 kB)
Collecting typing-extensions>=3.10.0 (from scikit-build-core)
Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.3->scikit-build-core)
Using cached zipp-3.19.2-py3-none-any.whl.metadata (3.6 kB)
Using cached scikit_build_core-0.9.6-py3-none-any.whl (152 kB)
Using cached exceptiongroup-1.2.1-py3-none-any.whl (16 kB)
Using cached importlib_resources-6.4.0-py3-none-any.whl (38 kB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached pathspec-0.12.1-py3-none-any.whl (31 kB)
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Using cached zipp-3.19.2-py3-none-any.whl (9.0 kB)
Installing collected packages: zipp, typing-extensions, tomli, pathspec, packaging, exceptiongroup, importlib-resources, scikit-build-core
Successfully installed exceptiongroup-1.2.1 importlib-resources-6.4.0 packaging-24.1 pathspec-0.12.1 scikit-build-core-0.9.6 tomli-2.0.1 typing-extensions-4.12.2 zipp-3.19.2
Installing build dependencies ... done
Running command Getting requirements to build wheel
Could not determine CMake version via --version, got '' 'Traceback (most recent call last):\n File "/home/kinovagaze/.local/bin/cmake", line 5, in <module>\n from cmake import cmake\nModuleNotFoundError: No module named \'cmake\'\n'
Getting requirements to build wheel ... done
Running command pip subprocess to install backend dependencies
Collecting cmake
Using cached cmake-3.29.5.1.tar.gz (30 kB)
ERROR: Exception:
Traceback (most recent call last):
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
return func(self, options, args)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 377, in run
requirement_set = resolver.resolve(
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
return bool(self._sequence)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
return any(self)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 182, in _make_candidate_from_link
base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 228, in _make_base_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 290, in __init__
super().__init__(
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
dist = self._prepare_distribution()
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 70, in _get_prepared_distribution
with build_tracker.track(req, tracker_id):
File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 137, in track
self.add(req, tracker_id)
File "/home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 103, in add
raise LookupError(message)
LookupError: https://files.pythonhosted.org/packages/1a/57/706da5e1855b720b709de64a12a85f36f6d6622acd4ac3f61a19448d6d42/cmake-3.29.5.1.tar.gz (from https://pypi.org/simple/cmake/) (requires-python:>=3.7) is already being built: cmake from https://files.pythonhosted.org/packages/1a/57/706da5e1855b720b709de64a12a85f36f6d6622acd4ac3f61a19448d6d42/cmake-3.29.5.1.tar.gz
error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/kinovagaze/catkin_ws/src/kinovagaze/gui/bin/python3 /home/kinovagaze/catkin_ws/src/kinovagaze/gui/lib/python3.8/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-iasp1wks/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- cmake 'ninja>=1.5'
cwd: [inherit]
Installing backend dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Seeing the same issue on SLES11 (old glibc version) and on SLES12 when using --no-binary. Seeing the following when running with --verbose:
Installing collected packages: pathspec, packaging, scikit-build-core
Successfully installed packaging-24.1 pathspec-0.12.1 scikit-build-core-0.9.6
Installing build dependencies ... done
Running command Getting requirements to build wheel
Could not determine CMake version via --version, got '' 'Traceback (most recent call last):\n File "venv/bin/cmake", line 5, in <module>\n from cmake import cmake\nModuleNotFoundError: No module named \'cmake\'\n'
Getting requirements to build wheel ... done
Running command pip subprocess to install backend dependencies
I'm in a venv with preexisting cmake package installed and upgrading to a newer version. The version warning message is coming from scikit-build-core so I'm suspecting something weird is going on here where the backend dep is opting into building wheel cmake rather than using the existing cmake binary for e.g. build_cmake entrypoint. But could be wrong just a guess from a quick cursory glance over the repo.
You can't install the cmake or ninja packages from binaries if you don't already have cmake and ninja on your system. You can install any system version, then it should build correctly. It can't bootstrap itself without a system cmake & ninja.
@stephen-hansen You'll want to disable build isolation if you need to use the cmake package in your venv when building a new cmake package. Build isolation makes a new venv that can't see the old one. I think it would work if you had cmake installed with pipx.
Hmm, actually, I wonder if we have a circular import issue here, though. cmake is careful not to require cmake, but ninja requires cmake and cmake can require ninja. I don't think it will break something that could have worked, but it might produce a weird error instead of telling you that you need to install a system ninja, and a system cmake.
You can't install the cmake or ninja packages from binaries if you don't already have cmake and ninja on your system. You can install any system version, then it should build correctly. It can't bootstrap itself without a system cmake & ninja.
@stephen-hansen You'll want to disable build isolation if you need to use the cmake package in your venv when building a new cmake package. Build isolation makes a new venv that can't see the old one. I think it would work if you had cmake installed with pipx.
Hmm, actually, I wonder if we have a circular import issue here, though. cmake is careful not to require cmake, but ninja requires cmake and cmake can require ninja. I don't think it will break something that could have worked, but it might produce a weird error instead of telling you that you need to install a system ninja, and a system cmake.
Yep that worked perfectly, thanks! That might solve the issue for other people here.
To be clear I only saw this when building with a preexisting cmake wheel in my PATH. Never saw this error with system cmake. So --no-build-isolation makes complete sense here.
I think the changes since previous versions are:
- The bootstrap CMake required is now 3.15+, while much older versions used to work
- Ninja is required, for some reason there's an issue if we fall back to make, so we disabled it for now.
- There's a circular issue if ninja isn't present causing a less helpful error
A few things we could do to help:
- Can we work with the make fallback? That would help keep ninja from being requested
- Maybe disable the ninja request entirely
- Could we support bootstrapping? (Would need (private?) support in scikit-build-core)
The problem with --no-build-isolation is that if cmake is just a (build-time) dependency in a chain of other dependencies, it's likely that along this dependency chain some other packages will stumble over it and the whole thing fails.
If system cmake needs to be quite new for the bootstrap to be successful, then users might have to rely on external repositories to get a binary for their distribution. In my case it was ubuntu:20.04 which ships cmake 3.16.3, but that still seems to be too old – as scikit-build still attempts an sdist build from PyPI. Unfortunately, for that distribution the cmake backport PPA is broken (it's missing a cmake-data provider) and installing cmake via apt.kitware.com PPA likewise fails because of this bug.
It's not supposed to, I'm assuming this is already installed? If someone manually places "cmake" in their requirements, then it will trigger this no matter what, but using a proper tool like scikit-build-core will only request cmake if not present.
We need to update our README, that would also help.
