All-Hands-AI
Add the ability to set LLM_BASE_URL with make setup-config
There are many people struggling to use OpenDevin with other LLMs (anything that isn't the official OpenAI API) https://github.com/OpenDevin/OpenDevin/issues/592 https://github.com/OpenDevin/OpenDevin/issues/417
This should make a little more straightforward to use local LLMS, or anything wich requires to set the URL
Does anything else than local requires a base url? I think LiteLLM forwards the others according to the model.
If so, it might be worth to say somewhere that BASE_URL is "only for local models", so that folks don't assume everyone needs something there. I'm not sure though.
Does anything else than local requires a base url? I think LiteLLM forwards the others according to the model.
If so, it might be worth to say somewhere that BASE_URL is "only for local models", so that folks don't assume everyone needs something there. I'm not sure though.
Yes, in most cases it's for local models, but it's also needed for OpenAI on Azure, for example. if not needed, you can just leave it blank and it won't be added to the config file
Maybe something like "[default: leave blank]" or "[default: ]" would be better?
That's great, is there any hint to tell the user what form BASE_URL should take? Frankly, I was quite confused at first as to whether it should be https://api.openai.com/v1/ or just https://api.openai.com/.
That's great, is there any hint to tell the user what form BASE_URL should take? Frankly, I was quite confused at first as to whether it should be https://api.openai.com/v1/ or just https://api.openai.com/.
You don't need it if you are using the official OpenAI API.
The url should have the /v1/, i tested it with Koboldcpp and it was working with the url set at http://localhost:5001/v1/
I added an hint wich should make it more clear