ParisNeo
Add a left bar where we can change the model parameters in realtime
Currently those options can be changed when calling the application:
--temp: the sampling temperature for the model (default: 0.1) --n-predict: the number of tokens to predict at a time (default: 128) --top-k: the number of top-k candidates to consider for sampling (default: 40) --top-p: the cumulative probability threshold for top-p sampling (default: 0.90) --repeat-penalty: the penalty to apply for repeated n-grams (default: 1.3) --repeat-last-n: the number of tokens to use for detecting repeated n-grams (default: 64) --ctx-size: the maximum context size to use for generating responses (default: 2048)
It would be a good thing to have a right bar that contains controls to set those parameters in the UI. If someone have the time to do this I'd apreciate it:
If you have time, can you add some sliders to select the temperature and so on. Just put them into a form with update button that sends this all to a /update_settings route on the flask server.
Tomorrow I work, and can't really have as much time as I have in weekends. Any help is welcome.