[User] Deadlock if number of threads > number of (hyper)threads

[jmtatsch]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [ x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [ x] I carefully followed the README.md.
• [ x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [ x] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I expect the program to run suboptimally but finish.

Current Behavior

Currently the program locks up with very large cpu utilization.

Environment and Context

I have a 6 core intel machine i.e. 12 threads with hyperthreading. Once I run with -t 13 the deadlock happens: ./main -m ./models/ggml-vicuna-13b-1.1-q4_0.bin -n 256 --repeat_penalty 1.1 --color -i -r "### Human:" -f prompts/chat-with-vicuna.txt -t 13

[syl-00110111]

effectivelly

[sw]

Are you sure it's a deadlock and not simply catastrophically slow? I get a slowdown by 150x when I use n_threads == cores + 1, but it still works.

[jmtatsch]

Maybe I didn't have the patience to really wait it out. However it wastes a ton of cpu cycles / energy.

[Green-Sky]

someone shot their foot.