text-generation-inference

text-generation-inference copied to clipboard

System Info

Hi all, I was installing from source and I got this error:

Building wheels for collected packages: vllm Building editable for vllm (pyproject.toml) ... error error: subprocess-exited-with-error

× Building editable for vllm (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [134 lines of output] running editable_wheel creating /tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info writing /tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info/PKG-INFO writing dependency_links to /tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info/dependency_links.txt writing requirements to /tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info/requires.txt writing top-level names to /tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info/top_level.txt writing manifest file '/tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file '/tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm.egg-info/SOURCES.txt' creating '/tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm-0.4.1+cu124.dist-info' creating /tmp/pip-wheel-21t7kkha/.tmp-uir5ykpr/vllm-0.4.1+cu124.dist-info/WHEEL running build_py running build_ext CMake Error: Could not find CMAKE_ROOT !!! CMake has most likely not been installed correctly. Modules directory not found in /tmp/pip-build-env-l25ha9ft/overlay/share/cmake-3.31 CMake Error: Could not find CMAKE_ROOT !!! CMake has most likely not been installed correctly. Modules directory not found in /tmp/pip-build-env-l25ha9ft/overlay/share/cmake-3.31 CMake Error: Error executing cmake::LoadCache(). Aborting.

  Traceback (most recent call last):
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 137, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 338, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 261, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 288, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 98, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "<string>", line 178, in build_extensions
    File "<string>", line 161, in configure
    File "/opt/conda/envs/tgi/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/home/ubuntu/Desktop/Code/text-generation-inference/server/vllm', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/tmpdfy9dco8.build-lib/vllm', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=/tmp/tmp8ga4ujxd.build-temp', '-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=/opt/conda/envs/tgi/bin/python3.11', '-DNVCC_THREADS=1', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=192']' returned non-zero exit status 1.
  /tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py:973: _DebuggingTips: Problem in editable installation.
  !!
  
          ********************************************************************************
          An error happened while installing `vllm` in editable mode.
  
          The following steps are recommended to help debug this problem:
  
          - Try to install the project normally, without using the editable mode.
            Does the error still persist?
            (If it does, try fixing the problem before attempting the editable mode).
          - If you are using binary extensions, make sure you have all OS-level
            dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
          - Try the latest version of setuptools (maybe the error was already fixed).
          - If you (or your project dependencies) are using any setuptools extension
            or customization, make sure they support the editable mode.
  
          After following the steps above, if the problem still persists and
          you think this is related to how setuptools handles editable installations,
          please submit a reproducible example
          (see https://stackoverflow.com/help/minimal-reproducible-example) to:
  
              https://github.com/pypa/setuptools/issues
  
          See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
          ********************************************************************************
  
  !!
    cmd_obj.run()
  Traceback (most recent call last):
    File "/opt/conda/envs/tgi/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/opt/conda/envs/tgi/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/conda/envs/tgi/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 273, in build_editable
      return hook(wheel_directory, config_settings, metadata_directory)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 475, in build_editable
      return self._build_with_temp_dir(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 406, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 319, in run_setup
      exec(code, locals())
    File "<string>", line 383, in <module>
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 183, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 137, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 338, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 261, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 288, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 98, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-l25ha9ft/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "<string>", line 178, in build_extensions
    File "<string>", line 161, in configure
    File "/opt/conda/envs/tgi/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/home/ubuntu/Desktop/Code/text-generation-inference/server/vllm', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/tmpdfy9dco8.build-lib/vllm', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=/tmp/tmp8ga4ujxd.build-temp', '-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=/opt/conda/envs/tgi/bin/python3.11', '-DNVCC_THREADS=1', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=192']' returned non-zero exit status 1.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building editable for vllm Failed to build vllm ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (vllm) make[1]: *** [Makefile-vllm:11: install-vllm-cuda] Error 1 make[1]: Leaving directory '/home/ubuntu/Desktop/Code/text-generation-inference/server' make: *** [Makefile:2: install-server] Error 2 (tgi) ubuntu@ip-172-31-28-135:~/Desktop/Code/text-generation-inference$

Information

• [ ] Docker
• [X] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

Installing from source

Expected behavior

Successful Installation of TGI