llama-cpp-python
llama-cpp-python copied to clipboard
abetlen
Python bindings for llama.cpp
``` jinja2.exceptions.TemplateSyntaxError: Encountered unknown tag 'generation'. Jinja was looking for the following tags: 'elif' or 'else' or 'endif'. The innermost block that needs to be closed is 'if'. ``` llama-cpp-python...
**Describe the solution you'd like** Support for PIL library image input (path) instead of Base64 encoding. For example, when using models with transformers library, I provide images this way `img...
**Is your feature request related to a problem? Please describe.** I make lots of use of these bindings, but frequently find that new models depend on changes in the upstream...
I am running llama-cpp-python Version: 0.3.16 Trying to load the recently released model [embeddinggemma-300M](https://huggingface.co/unsloth/embeddinggemma-300m-GGUF) I get the following error message: `llama_model_load: error loading model: error loading model architecture: unknown model...
Since this repo hasn’t been maintained in over 6 months and I couldn’t get in touch with the original author (@abetlen) via issues or socials, I’ve started a maintained fork:...
**Is your feature request related to a problem? Please describe.** Currently your documentation lists `minicpm-v-2.6` with `MiniCPMv26ChatHandler`. Since `MiniCPM-V 4.5` is out - could you please supprt it? **Describe the...
Hi, while reading the README, I found a few typos that I fixed. - `API to for basic` -> `API for basic` - `The model will will format` -> `The...
Equivalent to the `-ot` llama.cpp argument: ``` {"--override-tensor", "-ot"}, "=,...", ``` Can be passed as an optionlal string to the `Llama` class using the new `override_tensor` parameter. Same format as...
Updated to support Llama.cpp tags/b6490 fixes kv_cache errors lays groundwork for more future proof, raw, passthrough class which will interface more closely with llama.cpp
I'm trying to test llama-cpp-python (CPU mode) on SnapDragon X Plus processor with python 3.12. `pip install llama-cpp-python --prefer-binary --no-cache-dir --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu` and it is successfully installed. but, when i...
