S. Neuhaus

Did you enable CUDA in WSL? https://learn.microsoft.com/en-us/windows/ai/directml/gpu-cuda-in-wsl

Wouldn't it make more sense to keep this project minimal and have a gradio interface as a separate project?

OK, i also cannot get it to run with "torchrun", i get "failed to create process". Edit: It does work with Linux. The workaround for Windows with python 3.9.x is...

You need to provide more information about your OS, python, the libraries and their versions and your GPU.

nccl is not available on Windows. Switch to Linux or change "nccl" to "gloo" [here in example.py](https://github.com/facebookresearch/llama/blob/main/example.py#L23)

duplicate of #26

No, 30B is correct. At least the download works.

Try https://github.com/oobabooga/text-generation-webui it has such a feature. But yes beyond the max of 2048 you need to find workarounds.

The download script download.sh is not for windows.

Some systems have curl, some have wget. Ideally the script would use whatever is available.