Marie

Model Repeats Nonsensical Output

1

Hello, I need to run a gguf model on an embedded device with limited resources. I use the qwen2.5-0.5b-instruct-q4_0.gguf model. After testing a bunch of combinations, this is the llama-cli...