Pavol Rusnak

https://github.com/ggerganov/llama.cpp/pull/970 has been merged Please rebase the branch on top of current master: ``` git checkout master git pull git checkout quantize-tests git rebase master git push --force ``` or...

Btw, `sysctl hw.physicalcpu` returns 8 on M1, but you want to use only 4 threads, because M1 contains 4 high-performance cores and 4 low-performance cores.

> Are you able to report some rough results of running 8 v.s. 4 cores in terms of the ms per token for inference mode? 7B 4 threads => 80ms/token...

Btw, this is the output of `sysctl -a | grep hw.perflevel` on my M1: ``` hw.perflevel0.physicalcpu: 4 hw.perflevel0.physicalcpu_max: 4 hw.perflevel0.logicalcpu: 4 hw.perflevel0.logicalcpu_max: 4 hw.perflevel0.l1icachesize: 196608 hw.perflevel0.l1dcachesize: 131072 hw.perflevel0.l2cachesize: 12582912 hw.perflevel0.cpusperl2:...

> Anyone have a non-M1 mac? What is the result of grepping perflevel? I confirmed that `hw.perflevel0.physicalcpu` exists on Intel iMac and Intel Macbook too. So we can use that...

> In order not to result in poor default performance for corner cases like E/P cores, I decided to clip to max 4 for default, and warn the user if...

Thinking about this more ... Since we can reasonably well detect number of physical cores on Linux and macOS, I don't think we should be clamping the number of cores...

Getting an USB Vendor ID is expensive = $5000. Better option is to get a free Product ID under Vendor ID 0x1209 - see https://pid.codes and https://pid.codes/howto/ for more information

> a recent Trezor Model T firmware update What version of Trezor firmware are you using right now? Do you remember what version you were using before the update? Here's...

Worth checking out - can these be related? https://github.com/trezor/trezor-firmware/issues/2811