How to utalize more threads?

Open Leander-Andersen opened this issue 2 years ago • 1 comments

In the output after you exit the application, it says: '--seed 355555556 --threads 4 --n_predict 200 --model models/7B/ggml-model-q4_0.bin --top_k 40.' As you can see, it only uses 4 threads, and I have 12. How do I make it utilize all of them?

Apr 16 '23 21:04 Leander-Andersen

See in the upper right of the web gui, there you should be able to set the threads,

Apr 21 '23 05:04 supportend