llm-course icon indicating copy to clipboard operation
llm-course copied to clipboard

Fixing the Colab memory issue and llama.cpp/quantize script problem on CUDA

Open tolgakurtuluss opened this issue 1 year ago • 0 comments

Hi all,

Before I start to explain my pr, wanted to thank Maxime Labonne for this valuable content. I have been working on sources on the repository and liked each of them.

After trying to use AUTOGGUF repo, I faced with multiple problems such as Google Colab memory issue and llama.cpp/quantize script problem with CUDA version. I fixed both problems on the ".ipynb" file I uploaded below with adding related markdown explanations. If you directly merge it with the main repo, you'll notice that only 4-5 rows are added into jupyter notebook.

Hope that I can contribute such amazing repository as much as I can!

tolgakurtuluss avatar Feb 13 '24 17:02 tolgakurtuluss