Chat API broken by *is typing* messages

[poisson-fish]

The API got broken very recently with the addition of the is typing message. Now instead of returning the prompt result, all we get is is typing in the response datas:

{
  "data": [
   [["hello world","<em>Is typing...</em>"]], // first element is the prompt and second is the prompt result
  ],
  "duration": 0.0039386749267578125, // number of seconds to run function call
  "average_duration": 0.0009846687316894531
}

[oobabooga]

Ugh, that's bad.

[poisson-fish]

Yeah, I'm looking into it but there's a lot of stuff to look through so no promises

[oobabooga]

That only affects the API in chat mode, which is not completely supported yet. For instance, you cannot access the chat history or switch characters through the present API.

If you really want to use the chat API nevertheless, you can to it with streaming and discard the initial "Is tying..." message using this:

https://github.com/oobabooga/text-generation-webui/blob/main/api-example-stream.py

[poisson-fish]

Ah gotcha, I'll try the websocket option. Is a slightly more robust dedicated API separate from gradio something that could see a PR acceptance?

[oobabooga]

If you could do it through an extension, that would be great.

[troed]

Additionally the current example .py don't have all the parameters for the chat API - but as the REST version is broken maybe it's better to make it explicit it doesn't work? Came here to ask for which API endpoint to use to get the now asynchronous response :/

[oobabooga]

I have removed the chat API for now to avoid further confusion https://github.com/oobabooga/text-generation-webui/commit/72d207c0980232db287f9ce89ec4dd3b032465e5

[angelfor3v3r]

Also curious what the best course of action is to get a nice API with chat stuff. I'm guessing its just not finished yet/official?