cog icon indicating copy to clipboard operation
cog copied to clipboard

Provide OpenAI compatible HTTP endpoints

Open eburnette opened this issue 1 year ago • 3 comments

Currently the cog http server provides a REST interface with the "/predictions" endpoint. This is a request to provide OpenAI compatible endpoints. The goal would be to use a cog as a drop-in replacement for OpenAI and compatible models. Ideally, this would include embeddings and a chat interface (with and without streaming).

eburnette avatar Mar 24 '24 18:03 eburnette

have the same need

sinopec avatar Apr 02 '24 06:04 sinopec