Adding LM Studio Support

[ayoubachak]

Added the option to use local models via LM Studio, which gave decent results, here is an example of me asking mistral instruct 7B to make the game of life :

Note : I've set the context size to 32768 tokens even though most of them were not used, but I had some issue using other models like gemma 7b where it kept looping through the prompt infinitly.

[Ascendant-Stoic]

I downloaded the files you edited and copied them to their corresponding Devika folders and activated LM Studio local server but I still can't get it to work ... I even edited the Config.toml and added an this line --> LM_STUDIO = "http://localhost:1234/v1/chat/completions" to be able to edit LM_Studio API Endpoint in the settings and still nothing, what am I doing wrong?

[ayoubachak]

I downloaded the files you edited and copied them to their corresponding Devika folders and activated LM Studio local server but I still can't get it to work ... I even edited the Config.toml and added an this line --> LM_STUDIO = "http://localhost:1234/v1/chat/completions" to be able to edit LM_Studio API Endpoint in the settings and still nothing, what am I doing wrong?

Sorry for the late response, have you tried cloning my fork ? if so, can you share what's the issue exactly ? Also, can you first, try adding : LM_STUDIO = "http://localhost:1234/v1" instead of LM_STUDIO = "http://localhost:1234/v1/chat/completions" in the config.toml