Feature: Add support for custom models localhost e.g. TheBloke/WizardCoder-15B-1.0-GPTQ

[miguelgargallo]

trafficstars

Possibility to avoid using paid apis, and use TheBloke/WizardCoder-15B-1.0-GPTQ for example

I am sure here we all know this but I put the source in case someone don't know https://huggingface.co/TheBloke/WizardCoder-15B-1.0-GPTQ

[geekan]

code llama / llama2 / wizard coder will be integrated

[j-loquat]

Would be great to be able to assign multiple local models to the different agents. For example, run llama2 on gpu and assign it to CEO and other managers agents, and run a code-focused local model on cpu and assign it to the engineer agent, etc.

[j-loquat]

One thing to consider with local LLM agents is that we should keep the prompts shorter than for OpenAI and reduce the temperature to perhaps lower than 0.5. Lower temp and shorter prompts makes a huge difference in local response times as per GPT4All project.