llama2-webui icon indicating copy to clipboard operation
llama2-webui copied to clipboard

Published 20 hours ago • verified publisher icon liltom-eth

Very slow generation

Open jaslatendresse opened this issue 1 year ago • 1 comments

I am running this on Mac M1 16GB RAM using app.py for simple text generation. Using the llama.cpp from terminal is much faster but when I use the backend through app.py is very slow. Any ideas?

jaslatendresse avatar Dec 13 '23 18:12 jaslatendresse