ddh0

How to merge models of two different sizes and architectures?

5

Hello! I'm trying to merge a 7B mistral with a 13B Llama-2 model. In my mind I'd like to essentially keep the 7B model the same and just expand it...

[REQUEST] Add Llama 3 Instruct chat template

2

Hi! I've just started using Chat UI and I think it would be helpful to have the Llama 3 Instruct chat template + system prompt listed in PROMPTS.md so it...

[REQUEST] Accept raw token IDs in `stop` parameter

11

**Is your feature request related to a problem? Please describe.** I use `Llama.create_completion()` for my workflow, which allows me to pass stopping strings to end the generation. However, actually stopping...

improve GGUF metadata handling

4

- Add support for GGUFv2 (previously only GGUFv3 was supported) - Support execution of big-endian GGUF files on big-endian hosts - Fail if magic bytes `b'GGUF'` are wrong - Continue...

Support all types of GGUF metadata

1

This is a new PR after I had to force-push. See the previous PR for more details: #1497 @abetlen I would appreciate your input on this PR and any further...

support GLM-4.5V and GLM-4.1V vision models

11

Add support for [zai-org/GLM-4.5V](https://huggingface.co/zai-org/GLM-4.5V) and [zai-org/GLM-4.1V-9B-Thinking](https://huggingface.co/zai-org/GLM-4.1V-9B-Thinking) vision models to llama.cpp. I currently only plan to support images + text, no video inputs in this PR. The architecture is **`Glm4vMoeForConditionalGeneration`** (`"model_type":...