AssertionError: Torch not compiled with CUDA enabled

[Filmz5]

receiving this error after install

AssertionError: Torch not compiled with CUDA enabled

[clovistx]

Both ingest.py and run_localGPT.py is forcing cuda device. Maybe if you change these device you could run. I've tried on my Mac M1 to 'mps' but there is another error so i couldn't run the project yet.

[zhuolisam]

I faced the same issue on Windows. Running pip install -r requirements.txt might not install torch with cuda-toolkit. You have two options:

1. Use CPU by running python ingest.py --device_type cpu as mentioned in #6

2. Uninstall torch and reinstall torch with cuda-toolkit

#uninstall
conda uninstall pytorch
pip uninstall torch torchvision
pip uninstall torch torchvision #run this two times

#reinstall
pip install torch==2.0.0+cu118 torchvision==0.15.1+cu118 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118

[smithart]

Thanks, zhuolisam! This solution worked for me.

[mindwellsolutions]

I'm assuming you are running this on Windows 10/11, since I only saw this error on that not my Ubuntu machine. This step by step will install Cuda support for pytorch on windows.

https://pub.towardsai.net/installing-pytorch-with-cuda-support-on-windows-10-a38b1134535e