autoflow icon indicating copy to clipboard operation
autoflow copied to clipboard

protocol: show "Generating" status while it is really generating

Open sykp241095 opened this issue 1 year ago • 1 comments

If it is generating answer, use formal font-style, and prompting: "Generating answer..." image

sykp241095 avatar Oct 30 '24 08:10 sykp241095

This depends on a reproducible stream api.

As discussed before, we could use redis's XADD and XREAD to dispatch generating stream to multiple clients.

  • client 1 call chat api to generate chat
    • Server create a stream response to client 1
    • Server create a redis stream and duplicate each chunks from original stream
    • Optional Server store the stream to database or file storage for debug
  • client N call the new read stream api to read chunks from redis stream

634750802 avatar Dec 09 '24 03:12 634750802