Aleef Bilal

@kerlion What version of llama-cpp-python are you using? Can you also give me some inside about your platform (OS, etc).

@kerlion Great job on compiling it from source. Below is the command that might save you from the struggle of source compiling. `CMAKE_ARGS="-DGGML_CUDA=ON -DLLAMA_LLAVA=OFF" pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir`...

The issue still exists. Kindly report a fix if anyone finds it.

I bypassed this issue but using an early version of `llama.cpp` and then following the below temporary solution that has now become permanent it seems. `cd llama.cpp` `git checkout b3345`...

@IridiumMaster How do you install llama.cpp?

@IridiumMaster Below are the working steps for conversion of a vanilla model to `GGUF` using `Unsloth` and `llama.cpp`. The operation was performed using `RUNPOD` with `runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04` `1. git clone https://github.com/ggml-org/llama.cpp.git`...

@shamitv It is possible, but so far I've tested it with 0.2.9 and 0.3.4. I just want to know what could be the reasons behind this error. I've been using...

hey @mcglynnfinn Try installing the library using the below command `CMAKE_ARGS="-DGGML_CUDA=ON -DLLAMA_LLAVA=OFF" pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir `