alpaca-electron
alpaca-electron copied to clipboard
ItsPi3141
A setting to allow more cpu threads, if possible.
Is their a way for it to use more threads? I see it uses 8 but not more. I have 16 in total.
Performance will be worse if you use all 16 cores. Your OS will start to kinda die and it will make everything lag.
I am using 5900x, there is 8 more thread available, is it possible to allow more core? I want to use 22 out of the 24 threads available. Or is it need to be the power of two? like 8 16 32 threads?
I am using 5900x, there is 8 more thread available, is it possible to allow more core? I want to use 22 out of the 24 threads available. Or is it need to be the power of two? like 8 16 32 threads?
It works better with it's a power of 2. Back when llama.cpp just first came out, we tested it and found out that on a 32-core CPU, the performance, ranked from best to worst was the following:
- 16 cores
- 32 cores
- 24 cores
So it's better if it's a power of 2. And if it isn't a server (most people aren't running electron apps on servers), it will be slower if it uses all of the threads.
@ItsPi3141 - I've got a server with 60 cores and it shows as using 32/60 threads. Based on that performance test you listed, is 16 better than 32? My virtual host has 72 cores so technically I could allocate 64 threads to the app and still have plenty to spare for the OS; would that be better than 16 or 32? Or is going past 16 going to make things worse?
@ItsPi3141 - I've got a server with 60 cores and it shows as using 32/60 threads. Based on that performance test you listed, is 16 better than 32? My virtual host has 72 cores so technically I could allocate 64 threads to the app and still have plenty to spare for the OS; would that be better than 16 or 32? Or is going past 16 going to make things worse?
If your server/computer has 60 cores, it should use 32. If it has 72, then it should use 64.