noaebbot
Results
2
comments of
noaebbot
Got same error, after training and quantizing a llama3.1-8B. Read in the model fine, but when generating it got this error.
The error comes when i run model.generate. I have tried a variety of different cuda versions (12.1, 12.4, 12.6), but no one seems to work. Is it just that my...