apcameron
apcameron
Unfortunately I am not a Python coder. Not sure if this will help https://huggingface.co/blog/4bit-transformers-bitsandbytes
see https://github.com/imartinez/privateGPT/issues/203
See https://github.com/imartinez/privateGPT/issues/203
@ggerganov Do I need to make any more updates or is this PR good to go?
You may need this PR https://github.com/ggerganov/ggml/pull/218
@shusso It may be but I do not have enough RAM to try that model on my local computers.
I have not tried an earlier version as 2.0 is what I have installed on my system and it takes several hours compile on my small SBC. My test version...
I had to compile pytorch for my SBC as there are no builds currently for the RISCV64 platform for download. Are there specific options I need enabled when I compile...
I will try and compile pytorch 1.13.1 and see if that works for me on riscv64 with your code. I will let you know in the next day or so
@artyom-beilis Unfortunately pytorch 1.13.1 would not compile on my SBC so I am back to pytorch 2.x I have managed to get further but now it fails with this error...