FreeChat
FreeChat copied to clipboard
Allow to set -ngl setting
Would be great to have the option to set -ngl like in llama.cpp. Though there is GPU acceleration option, it does not seem to do much, as most of the work is done by the CPU. I have looked through the code to find that if GPU acceleration is enabled, the n_gpu_layers is set to 4, which is quite insubstantial. I have no experience nor knowledge about Swift yet, hence I failed to adjust the code after some attempts.
P.S.: I have tried setting n_gpy_layers from 4 to 99 (in AISettings View file) but it did not change anything. Also tried setting @AppSupport variable that would persist but to no luck.
By the way, great idea for Mac Users to try out the models and have an AI (especially accessible via a shortcut) at any time. Also, the sound of stopping the response generation is very satisfying! I wonder if a sound to sent a message could be changed to the same sound as the "stop generation" but a higher note.