llama-cpp-python icon indicating copy to clipboard operation
llama-cpp-python copied to clipboard

Published 20 hours ago • verified publisher icon abetlen

Feature: Lightweight llama_cpp.server Docker Image Build Workflow

Open devcxl opened this issue 10 months ago • 3 comments

By adding this workflow, the Docker image it builds becomes more lightweight. image You can see that the built image is one-tenth the size of the original image.

devcxl avatar Apr 05 '24 16:04 devcxl

docker run --network host --rm -it -v ./llama-server/:/data/ ghcr.io/devcxl/llama-cpp-python:light-server --host 0.0.0.0 --model /data/models/Qwen1.5/qwen1_5-0_5b-chat-q2_k.gguf By using this command, you can run it directly

devcxl avatar Apr 05 '24 17:04 devcxl

@Smartappli Okay, I will make these changes later today

devcxl avatar May 07 '24 06:05 devcxl

@abetlen  Can you approve testing plz?

Smartappli avatar May 23 '24 16:05 Smartappli