FreeChat icon indicating copy to clipboard operation
FreeChat copied to clipboard

Allow to set -ngl setting

Open normatovjj opened this issue 9 months ago • 4 comments

Would be great to have the option to set -ngl like in llama.cpp. Though there is GPU acceleration option, it does not seem to do much, as most of the work is done by the CPU. I have looked through the code to find that if GPU acceleration is enabled, the n_gpu_layers is set to 4, which is quite insubstantial. I have no experience nor knowledge about Swift yet, hence I failed to adjust the code after some attempts.

P.S.: I have tried setting n_gpy_layers from 4 to 99 (in AISettings View file) but it did not change anything. Also tried setting @AppSupport variable that would persist but to no luck.

By the way, great idea for Mac Users to try out the models and have an AI (especially accessible via a shortcut) at any time. Also, the sound of stopping the response generation is very satisfying! I wonder if a sound to sent a message could be changed to the same sound as the "stop generation" but a higher note.

normatovjj avatar May 23 '24 13:05 normatovjj