autoflow
autoflow copied to clipboard
protocol: show "Generating" status while it is really generating
If it is generating answer, use formal font-style, and prompting: "Generating answer..."
This depends on a reproducible stream api.
As discussed before, we could use redis's XADD and XREAD to dispatch generating stream to multiple clients.
-
client 1call chat api to generate chat- Server create a stream response to
client 1 - Server create a redis stream and duplicate each chunks from original stream
- Optional Server store the stream to database or file storage for debug
- Server create a stream response to
-
client Ncall the new read stream api to read chunks from redis stream