apex
apex copied to clipboard
Unable to build without CUDA support
Describe the Bug
The build fails if I don't have CUDA installed.
Minimal Steps/Code to Reproduce the Bug
$ pip install -v --disable-pip-version-check --no-cache-dir ... ./
Using pip 22.1.2 from /Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip (python 3.9)
Non-user install due to --prefix or --target option
Ignoring indexes: https://pypi.org/simple
Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-ephem-wheel-cache-hy72byyf
Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Initialized build tracking at /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Created build tracker: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Entered build tracker: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-install-qmo1wu_m
Processing /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src
Added file:///private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src to build tracker '/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg'
Running setup.py (path:/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py) egg_info for package from file:///private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src
Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-pip-egg-info-_dqz8haw
Preparing metadata (setup.py): started
Running command python setup.py egg_info
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py", line 88, in <module>
_, bare_metal_major, bare_metal_minor = get_cuda_bare_metal_version(CUDA_HOME)
File "/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py", line 16, in get_cuda_bare_metal_version
raw_output = subprocess.check_output([cuda_dir + "/bin/nvcc", "-V"], universal_newlines=True)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Warning: Torch did not find available GPUs on this system.
If your intention is to cross-compile, this is not an error.
By default, Apex will cross-compile for Pascal (compute capabilities 6.0, 6.1, 6.2),
Volta (compute capability 7.0), Turing (compute capability 7.5),
and, if the CUDA version is >= 11.0, Ampere (compute capability 8.0).
If you wish to cross-compile for a single specific architecture,
export TORCH_CUDA_ARCH_LIST="compute capability" before running setup.py.
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/python-3.9.13-j5uwi4faknh6e6wukglkq7djfq5wbo4j/bin/python3.9 -c '
exec(compile('"'"''"'"''"'"'
# This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
#
# - It imports setuptools before invoking setup.py, to enable projects that directly
# import from `distutils.core` to work with newer packaging standards.
# - It provides a clear error message when setuptools is not installed.
# - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
# setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
# manifest_maker: standard file '"'"'-c'"'"' not found".
# - It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute `setup.py` since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)
__file__ = %r
sys.argv[0] = __file__
if os.path.exists(__file__):
filename = __file__
with tokenize.open(__file__) as f:
setup_py_code = f.read()
else:
filename = "<auto-generated setuptools caller>"
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-pip-egg-info-_dqz8haw
cwd: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/
Preparing metadata (setup.py): finished with status 'error'
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Exception information:
Traceback (most recent call last):
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata
call_subprocess(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
raise error
pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
return func(self, options, args)
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 341, in run
requirement_set = resolver.resolve(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 75, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 506, in collect_root_requirements
req = self._make_requirement_from_install_req(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in _make_requirement_from_install_req
cand = self._make_candidate_from_link(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 215, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 291, in __init__
super().__init__(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
self.dist = self._prepare()
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in _prepare
dist = self._prepare_distribution()
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 428, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 497, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
self.req.prepare_metadata()
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 532, in prepare_metadata
self.metadata_directory = generate_metadata_legacy(
File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 71, in generate_metadata
raise MetadataGenerationFailed(package_details=details) from error
pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed
Removed file:///private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src from build tracker '/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg'
Removed build tracker: '/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg'
Expected Behavior
I would not expect the build to try to run nvcc if torch.cuda.is_available() is False. Even if it did, I would expect it to catch that error.
Environment
- macOS 12.6
- Python 3.9.13
- PyTorch 1.12.1