Alexandre Strube

> am at 272/300 gb right now. excitement starts to kick in, lets hope this thing runs. > only having 6x 4090 (144GB VRAM) and 512GB RAM, if this isnt...

This software seems abandoned.

It would be great to update docutil, too. This is an ancient version, and there's no reason to stop at 20.0

@Jerry-Ma I don't even see this "mount_fuse-ext2" available. Should we join this issue with a similar one, https://github.com/alperakcan/fuse-ext2/issues/34 ?

This doesn't work in a multi-gpu setup because all the multiple MPI instances of server.py try to bind to the web port and will fail. ```python srun --nodes=1 --cpus-per-task 16...

Seems like you are missing some parameter on the command line. Maybe you can edit your comment and show how you loaded it?

@tdguchi any news here? Did you try with the latest versions of everything? It was quite a while ago, perhaps the underlying libraries fixed it. Did you try with vllm?...

As @zhisbug said, there's a fix for using xformers instead. Which wouldn't force a full upgrade to Pytorch2 , which is VERY undesirable in many big systems.

@OrionRandD one can set it on the fastchat/serve/gradio_web_server.py Is this still an issue for you? Perhaps we could close this one if it isn't.

@Jeffwan hey, have you managed to do so? It's been a bit already. By the way, this is a FSDP bug, right? We should close the issue here and check...