OpenAI API compatibility

Open harishd1998 opened this issue 10 months ago • 2 comments

Currently the endpoint for embedding models is /embeddings on the infinity server, but I believe the correct open AI format would be /v1/embeddings ? correct me if Im wrong.

Feb 26 '25 13:02 harishd1998

Currently the endpoint for embedding models is /embeddings on the infinity server, but I believe the correct open AI format would be /v1/embeddings ? correct me if Im wrong.

--url-prefix /v1

Feb 28 '25 09:02 HyperForce

Some people will run it in a canary deployment or behind a load balancer for the /v1 prefix. If you run Infinity outside of k8s, just use --url-prefix /v1. You can see all options available in --help command.

Mar 01 '25 01:03 michaelfeil