Andrei
Andrei
I would love this
> > This issue is not meant to discuss the choice of scheme, please keep such discussion out of this issue. > > Where is such discussion intended to occur?...
I'm feeling this as well. Running llama-cpp manually is noticeably faster than running it through the API.
This license agreement is not acceptable for supposedly "free" software. It completely undermines the user and depending on what restrictions are put in place, has the potential to unnecessarily restrict...
> What about license - we can change filter hyperparameters and find balance for our community - this release under non-commercial license to get feedback and create commercial and free...
@mcmonkey4eva do you mean to say that modifications to the filter will be allowed in the license of the commercially compatible version?
@mayankchhabra I think this is a bug. I am having the same issue
I am running on linux on a ryzen 7 3700X, 32 gigabytes of ram.
Everything's also running extremely slow. Here's a screen record of what a simple generation looks like. [Screencast from 2023-08-18 10-31-58.webm](https://github.com/getumbrel/llama-gpt/assets/51296839/ffceccb0-24fb-488b-8226-bf093820d407) it took almost 3 minutes
I think it is good as it is now. It would not be elegant or make sense to have a separate API option for this.