ChatDoctor
ChatDoctor copied to clipboard
Kent0n-Li
gpu_count 0 | Cuda issue
As you can see im in Conda Powershell as Admin. I have installed PyTorch 2.0 with the updated torchvision for acceleration, along with the required downloads for transformers and tokenizer. The models load as well from the pretrained folder. Additionally, I have installed the CUDA toolkit 11.7 with drivers, and my GTX 1060 GPU 6GVRam is listed as available for computing. However, when attempting to activate CUDA, it shows as 0 or false. I am currently in the correct Conda environment and CUDA is installed and activated, but the issue persists. I noticed in the chat.py file that the model tokenizer shows as 8-bit floats to be disabled, which leads me to wonder if it is related. Also to mention that the tokenizers LLama name is written falsely perhaps, because i have found a thread on github on it, here is the link https://github.com/treadon/llama-7b-example/issues/1#issuecomment-1470007121 . There may be a typo error in your code as well in the chat.py file. I have been working on this issue for 3 days and would greatly appreciate any help. Thank you.
Look at line 13 in chat.py.
os.environ["CUDA_VISIBLE_DEVICES"]="1"
That line is telling the script to only allow GPU 1 to be used, but from the sounds of it you only have a single GPU. So you need to change it to use "0" for the CUDA_VISIBLE_DEVICE, or comment the line out (I'm not sure of the other consequences of that).
Look at line 13 in chat.py.
os.environ["CUDA_VISIBLE_DEVICES"]="1"That line is telling the script to only allow GPU 1 to be used, but from the sounds of it you only have a single GPU. So you need to change it to use "0" for the CUDA_VISIBLE_DEVICE, or comment the line out (I'm not sure of the other consequences of that).
thank you very much, the gpu works now, just out of memory by allocation, the trained model is apparently too voluminous to take in cache, i only have 6G Vram, but thank you again, a good thing to know