chat-ui icon indicating copy to clipboard operation
chat-ui copied to clipboard

Screen Rendering Slows Down Towards the End of Streaming from LLM Server

Open calycekr opened this issue 6 months ago • 7 comments

While the initial part of streaming from the LLM server is fine, the screen display speed slows down as time progresses, particularly towards the end of the streaming process. However, the LLM server has already finished sending the data, and only the screen display continues to be updated.

https://github.com/huggingface/chat-ui/blob/6de97af071c69aa16e8f893adebb46f86bdeeaff/src/routes/conversation/%5Bid%5D/%2Bserver.ts#L390-L396

I suspect the above code section. Could it be a shortage of buffer?

calycekr avatar Aug 06 '24 05:08 calycekr