FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

How deepseekcoder6.7b supports streaming output results

Open 469981325 opened this issue 10 months ago • 0 comments

How deepseekcoder6.7b supports streaming output results？ image

469981325 avatar Apr 19 '24 03:04 469981325

Similar issue here using Qwen-7B-Chat. What's the proper way to use streaming mode in openai api? Streaming for this model works fine in gradio web server.

This PR introduces the feature, but the test code uses from fastchat import Client, which does not exist.

Any advice? @baradm100

httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)

un-certainty avatar Apr 23 '24 04:04 un-certainty