Deepspeed Error on Windows - LINK : fatal error LNK1181: cannot open input file 'aio.lib'

[lolxdmainkaisemaanlu]

(textgen) C:\Users\Siddhesh\Desktop\text-generation-webui>pip install -r requirements.txt Collecting accelerate==0.15.0 Using cached accelerate-0.15.0-py3-none-any.whl (191 kB) Collecting bitsandbytes==0.36.0.post2 Using cached bitsandbytes-0.36.0.post2-py3-none-any.whl (76.3 MB) Collecting deepspeed==0.8.0 Using cached deepspeed-0.8.0.tar.gz (749 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [15 lines of output] test.c LINK : fatal error LNK1181: cannot open input file 'aio.lib' Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\Siddhesh\AppData\Local\Temp\pip-install-plr1qrri\deepspeed_02a1e689bf514f02a58b413d2ee0b723\setup.py", line 156, in abort(f"Unable to pre-compile {op_name}") File "C:\Users\Siddhesh\AppData\Local\Temp\pip-install-plr1qrri\deepspeed_02a1e689bf514f02a58b413d2ee0b723\setup.py", line 48, in abort assert False, msg AssertionError: Unable to pre-compile async_io DS_BUILD_OPS=1 [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] One can disable async_io with DS_BUILD_AIO=0 [ERROR] Unable to pre-compile async_io [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

[oobabooga]

On Linux it's necessary to install aio with

sudo apt install libaio-dev

https://github.com/oobabooga/text-generation-webui/issues/40#issuecomment-1412038622

So I guess on Windows it also has to be installed manually.

[ghost]

It looks like libaio isn't available at all on Windows. Could you perhaps try:

set DS_BUILD_AIO=0
set DS_BUILD_SPARSE_ATTN=0
pip install wheel
pip install -r requirements.txt

@oobabooga, seeing this do you think it would be wiser to take DeepSpeed out of requirements.txt and have it as an optional install? Shouldn't cause an import error unless --deepspeed is used.

[oobabooga]

@81300 DeepSpeed is a very small dependency (less than 1 MB). I think that's it's fine to leave it as a requirement even if it won't be used.

At some point it will be necessary to have a DeepSpeed guide in the wiki though. I can't write it now because I haven't really absorbed yet what it does and how to use it.

[ghost]

I think the problem is that upon installation DeepSpeed tries to compile stuff that won't work on Windows, even if you use the PyPI package. See https://github.com/microsoft/DeepSpeed/issues/2427 and how the official build script makes it work.

[lolxdmainkaisemaanlu]

I think the problem is that upon installation DeepSpeed tries to compile stuff that won't work on Windows, even if you use the PyPI package. See microsoft/DeepSpeed#2427 and how the official build script makes it work.

Yeah that's the issue, with the deepspeed commits now most users not familiar with the installation process would get stuck when installing the requirements.txt and be unable to use the webui at all..

Also btw, after a lot of effort, I did manage to compile Deepspeed (0.8.1) on Windows and got a .whl in the /dist directory. But the webui still fails to run on Windows because it gives this hostname -I error.

Before that it was giving mpi4py module not found error.. Seems like some backend stuff of Deepspeed doesn't work in Windows.

[gramster]

Also btw, after a lot of effort, I did manage to compile Deepspeed (0.8.1) on Windows and got a .whl in the /dist directory. But the webui still fails to run on Windows because it gives this hostname -I error.

Any chance you could share how you did that or share your wheel? I would like to try get CodeGeeX running on Windows and deepspeed is my current blocker.

[JeffMII]

Has anyone figured anything out? Why would Microsoft not support Windows in their own project?

[github-actions[bot]]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.

[marcanex]

So I guess only inference works with windows currently...I could be wrong as im as yet not exactly well versed in this area...but as far as i understand it...it should def help with text generation speed if not with gpu mem use.