apex icon indicating copy to clipboard operation
apex copied to clipboard

Unable to build without CUDA support

Open adamjstewart opened this issue 3 years ago • 0 comments

Describe the Bug

The build fails if I don't have CUDA installed.

Minimal Steps/Code to Reproduce the Bug

$ pip install -v --disable-pip-version-check --no-cache-dir ... ./
Using pip 22.1.2 from /Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip (python 3.9)
Non-user install due to --prefix or --target option
Ignoring indexes: https://pypi.org/simple
Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-ephem-wheel-cache-hy72byyf
Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Initialized build tracking at /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Created build tracker: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Entered build tracker: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg
Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-install-qmo1wu_m
Processing /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src
  Added file:///private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src to build tracker '/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg'
  Running setup.py (path:/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py) egg_info for package from file:///private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src
  Created temporary directory: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-pip-egg-info-_dqz8haw
  Preparing metadata (setup.py): started
  Running command python setup.py egg_info
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py", line 88, in <module>
      _, bare_metal_major, bare_metal_minor = get_cuda_bare_metal_version(CUDA_HOME)
    File "/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py", line 16, in get_cuda_bare_metal_version
      raw_output = subprocess.check_output([cuda_dir + "/bin/nvcc", "-V"], universal_newlines=True)
  TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

  Warning: Torch did not find available GPUs on this system.
   If your intention is to cross-compile, this is not an error.
  By default, Apex will cross-compile for Pascal (compute capabilities 6.0, 6.1, 6.2),
  Volta (compute capability 7.0), Turing (compute capability 7.5),
  and, if the CUDA version is >= 11.0, Ampere (compute capability 8.0).
  If you wish to cross-compile for a single specific architecture,
  export TORCH_CUDA_ARCH_LIST="compute capability" before running setup.py.

  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/python-3.9.13-j5uwi4faknh6e6wukglkq7djfq5wbo4j/bin/python3.9 -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-pip-egg-info-_dqz8haw
  cwd: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src/
  Preparing metadata (setup.py): finished with status 'error'
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Exception information:
Traceback (most recent call last):
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata
    call_subprocess(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 341, in run
    requirement_set = resolver.resolve(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 75, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 506, in collect_root_requirements
    req = self._make_requirement_from_install_req(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in _make_requirement_from_install_req
    cand = self._make_candidate_from_link(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 215, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 291, in __init__
    super().__init__(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
    self.dist = self._prepare()
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in _prepare
    dist = self._prepare_distribution()
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 428, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 497, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 532, in prepare_metadata
    self.metadata_directory = generate_metadata_legacy(
  File "/Users/ajstewart/spack/opt/spack/darwin-monterey-m1/apple-clang-14.0.0/py-pip-22.1.2-5z4p7jkcuzizwhyg4i5skiacffmslutj/lib/python3.9/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 71, in generate_metadata
    raise MetadataGenerationFailed(package_details=details) from error
pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed
Removed file:///private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-torch-nvidia-apex-master-yd4zh2x77wtlbfxlweg4hejq3lrusfpv/spack-src from build tracker '/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg'
Removed build tracker: '/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/pip-build-tracker-sjom3hlg'

Expected Behavior

I would not expect the build to try to run nvcc if torch.cuda.is_available() is False. Even if it did, I would expect it to catch that error.

Environment

  • macOS 12.6
  • Python 3.9.13
  • PyTorch 1.12.1

adamjstewart avatar Oct 01 '22 20:10 adamjstewart