tensorchord
Can't bypass OpenAI urls
I got modelz running (haven't gotten openmodelz running yet) but it looks like "import openai" poisons something, not sure what, and complain that "any" is not the password, as well as endpoints have changed. Inevitably I'm rediscovering the wheel here and it's already been discussed, perhaps project abandoned, but I thought I should ask as I cannot find the "truth" in reasonable time. Also, has modelz been kinda superseded by some other project?
Hi we do not have the bandwidth to maintain it. Maybe you could take a look at vLLM?
@gaocegege Btw, are you still operating modelz.ai? :) (When I try to sign up for the hosted LLM, I get timeout.)
If not, do you know a comparable service (private LLM in the cloud with OpenAI compatible (embedding) API)?
@Boscop Sorry for that. We do not support modelz.ai now. You could have a look at https://lepton.ai, https://modal.com, https://replicate.com