AgentGPT icon indicating copy to clipboard operation
AgentGPT copied to clipboard

Published 20 hours ago • verified publisher icon reworkd

Compatibility with local models / Expose OPENAI_API_HOST

Open jmtatsch opened this issue 2 years ago • 3 comments

Hello,

I would love to run AgentGPT with local models using https://github.com/abetlen/llama-cpp-python which provides an emulated openai api.

Can the API host be made user settable via a OPENAI_API_HOST env variable?

jmtatsch avatar Apr 12 '23 18:04 jmtatsch

i just want the same and maybe can we use https://github.com/cocktailpeanut/dalai as llama api..

matigumma avatar Apr 13 '23 16:04 matigumma

@matigumma Dalai does not provide a openai compatible api. The openai api is a nice abstraction over all the models behaving slightly differently.

jmtatsch avatar Apr 14 '23 07:04 jmtatsch

i made my own test here: https://github.com/matigumma/bb.agi/blob/main/baby_agi%20.ipynb

matigumma avatar Apr 14 '23 11:04 matigumma

Hey @jmtatsch and @matigumma we added support for custom base url via an ENV for local development, I imagine thats what you mean by host (https://github.com/reworkd/AgentGPT/pull/543)

asim-shrestha avatar Jun 05 '23 01:06 asim-shrestha

Hello @matigumma @jmtatsch I will be closing this as it seems to be resolved please feel free to reopen it if you still have any other questions ?

jasangill1 avatar Jun 29 '23 22:06 jasangill1

I'm setting export OPENAI_API_BASE="http://localhost:8000", but it's still using OpenAI Servers. Am I doing something wrong?

Swiffers avatar Jan 04 '24 14:01 Swiffers