Please povide OpenAI-compatible API

[fishfree]

When I access this path, it says: But OpenAI says:

[pseudotensor]

Yes, @this is working on this https://github.com/h2oai/h2ogpt/pull/133

[jinqiupeter]

Seems #133 has been merged, but still got {"detail": "Not Found"} error . Or maybe I didn't run it correctly. It's not clear how to enable Openai-compliant API access, could you please elaborate, thank you!

[this]

For the h2ogpt_client API, please refer https://github.com/h2oai/h2ogpt/tree/main/client#usage

[jinqiupeter]

Thanks @this . My understanding is that h2ogpt_client is a client that calls the backend h2ogpt's API, I don't need to run h2ogpt_client to get the backend API to work, right?

What I am trying to do is to use https://github.com/Yidadaa/ChatGPT-Next-Web to connect my h2ogpt backend. So far I've only got 404:

[jinqiupeter]

Btw here is how I run the h2ogpt backend:

export ALLOW_API=1
python3 generate.py --base_model=$MODEL --langchain_mode=ChatLLM --visible_langchain_modes="['ChatLLM', 'UserData', 'MyData']" --score_model=None --max_max_new_tokens=2048 --max_new_tokens=512 --infer_devices=False --load_8bit=True --share=True

$MODEL is h2oai/h2ogpt-gm-oasst1-en-2048-falcon-40b-v2

[this]

My understanding is that h2ogpt_client is a client that calls the backend h2ogpt's API, I don't need to run h2ogpt_client to get the backend API to work, right?

@jinqiupeter h2oGPT server API is not similar or compatible with OpenAI API, because h2oGPT HTTP API is implemented through the Gradio server. Hence, it would be much easier for you use the h2ogpt_client which provides a more user-friendly API.

[jinqiupeter]

Thank you, let me try tomorrow.

[enriquedecote]

Did you get this working @jinqiupeter ? I'm on a similar boat. I'd like to call the backend from a different frontend, maybe a Discord server or the next-web frontend you cite.

AFAIK you should be able to reach the API documentation through something like http://0.0.0.0:7860/?view=api but I just get a blank square with blurred background.

[pseudotensor]

done