Can't install Deepspeed

[AndreyRGW]

 pip install deepspeed
Collecting deepspeed
  Using cached deepspeed-0.6.5.tar.gz (567 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  python setup.py egg_info did not run successfully.
  exit code: 1

  [24 lines of output]
  The system cannot find the file specified.
  The system cannot find the file specified.
  The system cannot find the file specified.
  The system cannot find the file specified.
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "C:\Users\RGWyo\AppData\Local\Temp\pip-install-pdq_etar\deepspeed_5154de904eb44c55af7b10637df57bda\setup.py", line 152, in <module>
      abort(f"Unable to pre-compile {op_name}")
    File "C:\Users\RGWyo\AppData\Local\Temp\pip-install-pdq_etar\deepspeed_5154de904eb44c55af7b10637df57bda\setup.py", line 48, in abort
      assert False, msg
  AssertionError: Unable to pre-compile sparse_attn
  DS_BUILD_OPS=1
   [WARNING]  cpu_adam requires the 'lscpu' command, but it does not exist!
   [WARNING]  cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
   [WARNING]  cpu_adam requires the 'lscpu' command, but it does not exist!
   [WARNING]  cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
   [WARNING]  cpu_adagrad requires the 'lscpu' command, but it does not exist!
   [WARNING]  cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
   [WARNING]  cpu_adagrad requires the 'lscpu' command, but it does not exist!
   [WARNING]  cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
   [WARNING]  please install triton==1.0.0 if you want to use sparse attention
   [WARNING]  One can disable sparse_attn with DS_BUILD_SPARSE_ATTN=0
   [ERROR]  Unable to pre-compile sparse_attn
  [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Encountered error while generating package metadata.

See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I have Win11

Tried to disable sparse and async_io.

[mrwyattii]

@AndreyRGW sorry the late response on this. Can you try installing Triton and try again? pip install triton==1.0.0

[AndreyRGW]

@AndreyRGW sorry the late response on this. Can you try installing Triton and try again? pip install triton==1.0.0

Hello. First of all, my python doesn't see triton above version 0.3.0. Secondly, when I install triton 0.3.0, I get an error.

pip install triton
Collecting triton
  Using cached triton-0.3.0.tar.gz (920 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\RGWyo\AppData\Local\Temp\pip-install-prdcgekx\triton_bb4f915644654966978cabf00430a945\setup.py", line 98, in <module>
          find_llvm()
        File "C:\Users\RGWyo\AppData\Local\Temp\pip-install-prdcgekx\triton_bb4f915644654966978cabf00430a945\setup.py", line 28, in find_llvm
          raise RuntimeError('Could not find llvm-config. ' + instructions)
      RuntimeError: Could not find llvm-config. Please install llvm-{8, 9, 10}-dev
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[AndreyRGW]

pip index versions triton
WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
triton (0.3.0)
Available versions: 0.3.0, 0.2.3, 0.2.2, 0.2.1, 0.2.0, 0.1.3, 0.1.2, 0.1.1, 0.1

[mrwyattii]

@AndreyRGW try updating pip ( pip install --upgrade pip) and you should see the latest versions of triton:

venv ❯ pip index versions triton
triton (1.1.1)
Available versions: 1.1.1, 1.1.0, 1.0.0, 0.4.2, 0.4.1, 0.3.0, 0.2.3, 0.2.2, 0.2.1, 0.2.0, 0.1.3, 0.1.2, 0.1.1, 0.1
  INSTALLED: 1.0.0
  LATEST:    1.1.1

[AndreyRGW]

@AndreyRGW try updating pip ( pip install --upgrade pip) and you should see the latest versions of triton:

venv ❯ pip index versions triton
triton (1.1.1)
Available versions: 1.1.1, 1.1.0, 1.0.0, 0.4.2, 0.4.1, 0.3.0, 0.2.3, 0.2.2, 0.2.1, 0.2.0, 0.1.3, 0.1.2, 0.1.1, 0.1
  INSTALLED: 1.0.0
  LATEST:    1.1.1

I am using the latest version of pip (22.2.2), but triton 1.0.0 still does not appear.

[mrwyattii]

@AndreyRGW I just re-read your original post and noted that you're on Windows 11. I believe that's why the versions of triton differ in our pip output.

I don't currently have access to a system with Windows and GPUs, so I can't test the install process completely... but in powershell, I can avoid the error you are seeing by disabling pre-compile altogether:

$env:DS_BUILD_OPS = 0
pip install deepspeed

Our support for Windows is something we're working on, so if this doesn't work you could also try using Windows Subsystem for Linux (WSL) and install deepspeed in the linux environment that provides.

[yuanzhiyong1999]

Have you solved this problem?

[AndreyRGW]

Have you solved this problem?

On Windows? No. But it works fine on Ubuntu 22.04.

[yuanzhiyong1999]

Have you solved this problem?

On Windows? No. But it works fine on Ubuntu 22.04.

Thank you！

[PleezDeez]

Triton is now cracked for windows and can confirm that deepspeed package recognizes I have version 2.0.0 installed once moving the 2.0.0 build into the xformers folder and replacing any conflicts. I'll try the ds build ops command above when I get home from dinner

[PleezDeez]

It still warns about wanting triton 1.1.1 possibly causing issues but still picks it up being installed

[PleezDeez]

"DS_BUILD_OPS=0: The term 'DS_BUILD_OPS=0' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again."

and it was the same message both with spaces between the = and not

[PleezDeez]

I got past the cpu flag by installing py_cpuinfo but now im running into libaio-dev. I can use WSL to grab the libaio.so file it's looking for but I don't know where to put it back in windows in order to trick the deepspeed installer into thinking it's there

[PleezDeez]

Also I went file to file changing all the triton module stuff to 1.0.0 from 2.0.0 which was enough to stop that flag

[mrwyattii]

@PleezDeez You are seeing an error with the DS_BUILD_OPS environment variable because setting environment variables in Windows Command Prompt or PowerShell is different than on linux: Linux: export DS_BUILD_OPS=0 Command Prompt: set DS_BUILD_OPS=0 PowerShell: $Env:DS_BUILD_OPS=0

Please try setting DS_BUILD_OPS=0 and DS_BUILD_AIO=0 and you will not need to worry about libaio.so

[mrwyattii]

Closing due to inactivity, please reopen if the suggested environment variable settings do not work. Thanks.

[lanmower]

Due to inactivity? I just had this error, please re-open this issue.

[Syntax3rror404]

I also get the same error:

(venv) mzapf@aiworker:~$ DS_BUILD_OPS=1 pip install deepspeed
Collecting deepspeed
  Downloading deepspeed-0.8.2.tar.gz (763 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 763.5/763.5 KB 4.9 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [13 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-vhg78wwk/deepspeed_c3ae6d59752540238f6f4b52f0b123f3/setup.py", line 156, in <module>
          abort(f"Unable to pre-compile {op_name}")
        File "/tmp/pip-install-vhg78wwk/deepspeed_c3ae6d59752540238f6f4b52f0b123f3/setup.py", line 48, in abort
          assert False, msg
      AssertionError: Unable to pre-compile sparse_attn
      DS_BUILD_OPS=1
       [WARNING]  sparse_attn requires a torch version >= 1.5 but detected 2.0
       [WARNING]  using untested triton version (2.0.0), only 1.0.0 is known to be compatible
       [WARNING]  One can disable sparse_attn with DS_BUILD_SPARSE_ATTN=0
       [ERROR]  Unable to pre-compile sparse_attn
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[anni-mueller]

I also have this issue on ubuntu 22.04

triton 1.0.0 isnt aviable anymore

pip install triton==1.0.0
ERROR: Could not find a version that satisfies the requirement triton==1.0.0 (from versions: 2.0.0.dev20221030, 2.0.0.dev20221031, 2.0.0.dev20221101, 2.0.0.dev20221103, 2.0.0.dev20221105, 2.0.0.dev20221117, 2.0.0.dev20221120, 2.0.0.dev20221202, 2.0.0.dev20230208, 2.0.0.dev20230217, 2.0.0a1, 2.0.0a2, 2.0.0, 2.0.0.post1)
ERROR: No matching distribution found for triton==1.0.0

[vonplato]

Same problem in Windows 11

`pip install deepspeed Collecting deepspeed Using cached deepspeed-0.8.3.tar.gz (765 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [27 lines of output] No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1' test.c Microsoft (R) Incremental Linker Version 14.35.32216.1 Copyright (C) Microsoft Corporation. All rights reserved.

  LINK : warning LNK4044: unrecognized option '/L'; ignored
  LINK : fatal error LNK1181: cannot open input file 'C:PythonLibsite-packages.obj'
  'git' is not recognized as an internal or external command,
  operable program or batch file.
  'git' is not recognized as an internal or external command,
  operable program or batch file.
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "C:\Users\vonpl\AppData\Local\Temp\pip-install-dyziex_l\deepspeed_29dbd8be473f4dcbae7b33482fe3329b\setup.py", line 198, in <module>
      create_dir_symlink('..\\..\\csrc', '.\\deepspeed\\ops\\csrc')
    File "C:\Users\vonpl\AppData\Local\Temp\pip-install-dyziex_l\deepspeed_29dbd8be473f4dcbae7b33482fe3329b\setup.py", line 190, in create_dir_symlink
      os.remove(dest)
  PermissionError: [WinError 5] Access is denied: '.\\deepspeed\\ops\\csrc'
  [WARNING] Torch did not find cuda available, if cross-compiling or running with cpu only you can ignore this message. Adding compute capability for Pascal, Volta, and Turing (compute capabilities 6.0, 6.1, 6.2)
  DS_BUILD_OPS=0
   [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
   [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
   [WARNING]  sparse_attn cuda is not available from torch
   [WARNING]  sparse_attn requires a torch version >= 1.5 but detected 2.0
   [WARNING]  please install triton==1.0.0 if you want to use sparse attention
  Install Ops={'async_io': False, 'cpu_adagrad': False, 'cpu_adam': False, 'fused_adam': False, 'fused_lamb': False, 'quantizer': False, 'random_ltd': False, 'sparse_attn': False, 'spatial_inference': False, 'transformer': False, 'stochastic_transformer': False, 'transformer_inference': False, 'utils': False}
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.`

[McSash]

got the same error, any updates on this?

[AndreyRGW]

got the same error, any updates on this?

Well, I was able to compile deepspeed 0.8.3 for Windows last month.

[McSash]

Same error: `(venv) C:\Python\venv\Scripts>pip install deepspeed==0.8.3 Collecting deepspeed==0.8.3 Downloading deepspeed-0.8.3.tar.gz (765 kB) ---------------------------------------- 765.4/765.4 kB 4.4 MB/s eta 0:00:00 Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [19 lines of output] test.c LINK : fatal error LNK1181: Eingabedatei "aio.lib" kann nicht ge”ffnet werden. fatal: not a git repository (or any of the parent directories): .git Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\Sascha\AppData\Local\Temp\pip-install-_o9duc57\deepspeed_fbec153d955b4244ae7a50c9468c627f\setup.py", line 198, in create_dir_symlink('..\..\csrc', '.\deepspeed\ops\csrc') File "C:\Users\Sascha\AppData\Local\Temp\pip-install-_o9duc57\deepspeed_fbec153d955b4244ae7a50c9468c627f\setup.py", line 190, in create_dir_symlink os.remove(dest) PermissionError: [WinError 5] Zugriff verweigert: '.\deepspeed\ops\csrc' [WARNING] Torch did not find cuda available, if cross-compiling or running with cpu only you can ignore this message. Adding compute capability for Pascal, Volta, and Turing (compute capabilities 6.0, 6.1, 6.2) DS_BUILD_OPS=0 [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] sparse_attn cuda is not available from torch [WARNING] sparse_attn requires a torch version >= 1.5 but detected 2.0 [WARNING] please install triton==1.0.0 if you want to use sparse attention Install Ops={'async_io': False, 'cpu_adagrad': False, 'cpu_adam': False, 'fused_adam': False, 'fused_lamb': False, 'quantizer': False, 'random_ltd': False, 'sparse_attn': False, 'spatial_inference': False, 'transformer': False, 'stochastic_transformer': False, 'transformer_inference': False, 'utils': False} [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.`

I use Windows 11, which one do you use @AndreyRGW ? And how did you install deepspeed? I used cmd in Admin mode

[AndreyRGW]

Same error: `(venv) C:\Python\venv\Scripts>pip install deepspeed==0.8.3 Collecting deepspeed==0.8.3 Downloading deepspeed-0.8.3.tar.gz (765 kB) ---------------------------------------- 765.4/765.4 kB 4.4 MB/s eta 0:00:00 Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [19 lines of output] test.c LINK : fatal error LNK1181: Eingabedatei "aio.lib" kann nicht ge”ffnet werden. fatal: not a git repository (or any of the parent directories): .git Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\Sascha\AppData\Local\Temp\pip-install-_o9duc57\deepspeed_fbec153d955b4244ae7a50c9468c627f\setup.py", line 198, in create_dir_symlink('..\..\csrc', '.\deepspeed\ops\csrc') File "C:\Users\Sascha\AppData\Local\Temp\pip-install-_o9duc57\deepspeed_fbec153d955b4244ae7a50c9468c627f\setup.py", line 190, in create_dir_symlink os.remove(dest) PermissionError: [WinError 5] Zugriff verweigert: '.\deepspeed\ops\csrc' [WARNING] Torch did not find cuda available, if cross-compiling or running with cpu only you can ignore this message. Adding compute capability for Pascal, Volta, and Turing (compute capabilities 6.0, 6.1, 6.2) DS_BUILD_OPS=0 [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] sparse_attn cuda is not available from torch [WARNING] sparse_attn requires a torch version >= 1.5 but detected 2.0 [WARNING] please install triton==1.0.0 if you want to use sparse attention Install Ops={'async_io': False, 'cpu_adagrad': False, 'cpu_adam': False, 'fused_adam': False, 'fused_lamb': False, 'quantizer': False, 'random_ltd': False, 'sparse_attn': False, 'spatial_inference': False, 'transformer': False, 'stochastic_transformer': False, 'transformer_inference': False, 'utils': False} [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.`

I use Windows 11, which one do you use @AndreyRGW ? And how did you install deepspeed? I used cmd in Admin mode

I also have Windows 11 22h2, I did not install via pip, I compiled deepspeed from source. And as a terminal I use anaconda.

[McSash]

Thanks, with the steps provided here https://github.com/microsoft/DeepSpeed and afterwards pip install "deepspeed-0.9.1+cc67f22f-py3-none-any.whl" it finally worked for me

[zhengzzj]

@McSash Could you please tell me where can I download the file "deepspeed-0.9.1+cc67f22f-py3-none-any.whl"? Thank you. I could only find files of type .zip on the website https://github.com/microsoft/DeepSpeed/releases?page=1.

[Syntax3rror404]

I set the env DS_BUILD_SPARSE_ATTN=0 to install deepspeed. Triton == 1.0.0 is not available anymore for ubuntu 22.04

[liku-amare]

@PleezDeez You are seeing an error with the DS_BUILD_OPS environment variable because setting environment variables in Windows Command Prompt or PowerShell is different than on linux: Linux: export DS_BUILD_OPS=0 Command Prompt: set DS_BUILD_OPS=0 PowerShell: $Env:DS_BUILD_OPS=0

Please try setting DS_BUILD_OPS=0 and DS_BUILD_AIO=0 and you will not need to worry about libaio.so

Thanks, with the steps provided here https://github.com/microsoft/DeepSpeed and afterwards pip install "deepspeed-0.9.1+cc67f22f-py3-none-any.whl" it finally worked for me

Using these two installed deepspeed for me

[alex8430]

i got this error when i try to install package:

` × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [20 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/setup.py", line 38, in from op_builder.all_ops import ALL_OPS File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/op_builder/all_ops.py", line 29, in builder = get_accelerator().create_op_builder(member_name) File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/accelerator/mps_accelerator.py", line 220, in create_op_builder builder_class = self.get_op_builder(op_name) File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/accelerator/mps_accelerator.py", line 227, in get_op_builder from deepspeed.ops.op_builder.cpu import NotImplementedBuilder File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/deepspeed/init.py", line 21, in from . import ops File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/deepspeed/ops/init.py", line 6, in from . import adam File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/deepspeed/ops/adam/init.py", line 6, in from .cpu_adam import DeepSpeedCPUAdam File "/Users/zero/Desktop/ML-Trading-Models/DeepSpeed/deepspeed/ops/adam/cpu_adam.py", line 7, in from cpuinfo import get_cpu_info ModuleNotFoundError: No module named 'cpuinfo' [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.`

[CattleHome]

windows上，我在这里https://pypi.org/project/deepspeed/#files 下载了对应的包，解压之后直接放在虚拟环境里可以成功