dogjamboree

Performance is much worse than in llama.cpp on Apple Silicon (M1/M2)

10

Inference in general is much slower than with llama.cpp on my M2 Pro processor using any model (I tried Vicuna 13b, llama 13b, and alpaca 13b). I would say approximately...

Model does not obey follow up instructions very well(at all), compared to Vicuna for example

7

I use these LLM's for creative writing and with Vicuna I'm able to get it to revise sections, like add or remove details, make things longer or shorter, etc (with...