Alden Johnson

FP16 and 4-bit quantized model both produce garbage output on M1 8GB

1

Both the `ggml-model-q4_0` and `ggml-model-f16` produce a garbage output on my M1 Air 8GB, using the 7B LLaMA model. I've seen the quantized model having problems but I doubt the...