llama-cpp-python

llama-cpp-python copied to clipboard

`function_call` and `tool_choice` didn't have "auto" feature implemented

4

# Prerequisites Please answer the following questions for yourself before submitting an issue. - [x] I am running the latest code. Development is very rapid so there are no tagged...

MoeBuTa

RPC is broken due to change of interface in llama.cpp main repository (rpc : early register backend devices #11262)

4

RPC does not work in llama-cpp-python since this commit in [llama.cpp](https://github.com/ggerganov/llama.cpp) ``` Revision: 667d72846c06b2cf4f7c8a4265e210991a49706b Author: Radoslav Gerganov Date: 17/01/2025 09:57:09 Message: rpc : early register backend devices (#11262) Early register...

j-lag

# Prerequisites Please answer the following questions for yourself before submitting an issue. - [x] I am running the latest code. Development is very rapid so there are no tagged...

zhudotexe

pip install llama-cpp-python got stuck forever at "Configuring CMake" in docker

4

# Prerequisites Please answer the following questions for yourself before submitting an issue. - [ X] I am running the latest code. Development is very rapid so there are no...

jiafatom

trying to convert embedding into tensor leads to Segmentation fault: --- > System Info * Physical (or virtual) hardware you are using, e.g. for Linux: ``` > sysctl -a |...

devashishraj

* Physical (or virtual) hardware you are using, e.g. for Linux: ``` user@dev0:~$ lscpu Architecture: aarch64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7...

ekcrisp

When installing llama-cpp-python, I set CMAKE-ARGS="- DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS" but I don't seem to see any acceleration effect. What's the reason for this?

buptmengjj

I cannot install this package with cuda12.4

3

# Details - Windows 11 - NVIDIA CUDA 12.4 - NVIDIA GeForce RTX 4060 - Anaconda: Python 3.12.8 # Terminal Output ``` PS E:\OwnProject\aivtuber> $env:CMAKE_ARGS="-DGGML_CUDA=on" PS E:\OwnProject\aivtuber> pip install llama-cpp-python...

BoogonClothman

Failed building wheel for llama-cpp-python

6

Hello, I cannot install llama-cpp-python on Ubuntu 24.04.2, CUDA 12.4, python 3.12. Got issue copied below (I've deleted parts that were successful): (llms) patryk@patryk-MS-7E16:~/llama-cpp-python$ pip install llama-cpp-python \ --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124...

pklochowicz

[BUG] Server cant handle two streaming connections in same time

5

# Prerequisites Please answer the following questions for yourself before submitting an issue. - [x] I am running the latest code. Development is very rapid so there are no tagged...

ArtyomZemlyak