madmads11

I see that llama.cpp has added a [C-style API](https://github.com/ggerganov/llama.cpp/pull/370), exciting stuff!

> > I see that llama.cpp has added a [C-style API](https://github.com/ggerganov/llama.cpp/pull/370), exciting stuff! > > Yea. My bindings were based on my own C++ API ([#77](https://github.com/ggerganov/llama.cpp/pull/77) which is now closed)....

Would this support using and interacting with alpaca and llama models of all sizes?

I have a 3070 8GB VRAM and 32GB RAM and I am able to load the LLaMA 13b 4-bit model. In chat mode I can run a few messages back...

I was following the same guide as KamiAso, but I got the same error at the same point on Windows 11. Hopefully a solution to this will come as the...

> See if this helps: https://www.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_install_llama_8bit_and_4bit/ Yes! This works, specifically resuming the tutorial from step 6. > 6. Download [libbitsandbytes_cuda116.dll](https://github.com/DeXtmL/bitsandbytes-win-prebuilt) and put it in C:\Users\xxx\miniconda3\envs\textgen\lib\site-packages\bitsandbytes\ > 7. In \bitsandbytes\cuda_setup\main.py search...

Here is what I did to run Alpaca 30b on my system with llama.cpp. I would assume it would work with Alpaca 13b as well. 1. Downloaded and built llama.cpp...

I just saw the updated readme file stating that you cannot link to model downloads anywhere on this repository. Would instructions like mine, where in step 2 I link to...

> the ones you linked are sadly mixed, and not "pure" lora models. so i would assume no. you could just say "pi3141 alpaca 30B" model, and it would be...