llama-cpp-python

llama-cpp-python copied to clipboard

Can't install GPU version for windows for many times.

10

## Issues I am trying to install the lastest version of llama-cpp-python in my windows 11 with RTX-3090ti(24G). I have successfully installed llama-cpp-python=0.1.87 (can't exactly remember) months ago while using:...

XunfunLee

ValueError: could not broadcast input array from shape (371,) into shape (320,)

1

Hi, Getting this error when I am trying to summarize a document using llama-3 (via llama_cpp and llamaindex). I just found that it's a llama_cpp error and not LlamaIndex error....

anuragrawal2024

Crashing with "IndexError: index 200 is out of bounds for axis 0 with size 200"

9

Installed from conda environment with pip **Version: '0.2.69'** The code is as follow: ``` llm = Llama( model_path="/data/codelama-2024-02/CodeLlama-7b-Python/ggml-model-f16.gguf", seed=1023, # Uncomment to set a specific seed n_ctx=200, # Uncomment to...

phishmaster

### Discussed in https://github.com/abetlen/llama-cpp-python/discussions/1425 Originally posted by **VijayAsokkumar** May 3, 2024 Hi All, I am using llamacpppython in my app, which I have installed in a conda environment. I have...

VijayAsokkumar

Feature: Lightweight llama_cpp.server Docker Image Build Workflow

3

By adding this workflow, the Docker image it builds becomes more lightweight.  You can see that the built image is one-tenth the size of the original image.

devcxl

Add Nous Hermes 2 Pro function calling ChatHandler.

1

all is in the title 😉

stygmate

Unable to install llama-cpp-python with CUBLAS or CUDA enabled under tensorflow-gpu docker image.

1

I'm attempting to install llama-cpp-python under the tensorflow-gpu docker image (nightly build) . When I attempt to do so, I get the following error messages. ```` root@a1f1e127514b:/tf# CMAKE_ARGS="-DLLAMA_CUDA=on" FORCE_CMAKE=1 pip...

brent-halen

Failed to load shared library \venv\Lib\site-packages\llama_cpp\llama.dll

3

Hi, I am running llama-cpp-python on surface book 2 having i7 with nvidea geforce gtx 1060. I installed vc++, cuda drivers 12.4 Running on Python 3.11.3 Compiled llama using below...

mahesh557

Which Python + CUDA version do you recommend for LLaVA on Windows 10 Pro 64-bit?

1

Hey, I would appreciate recommendation on the specific versions of Python + CUDA environment on Windows that have been tested by you guys and proven to work well with LLaVAv1.6...

Vinventive

Failed to detect a default CUDA architecture

10

# Prerequisites Please answer the following questions for yourself before submitting an issue. - [x] I am running the latest code. Development is very rapid so there are no tagged...

arthurwolf