ai icon indicating copy to clipboard operation
ai copied to clipboard

What is the intended scope of this library?

Open bioshazard opened this issue 1 year ago • 0 comments

Request: Add a .md somewhere defining the intended limits of the features of this library.

I have adopted this library to deliver client/server solutions for handling basic LLM chat experience (loving it btw ty). I want to make sure I am outsourcing functionality wherever possible to limit my maintenance overhead. With that in mind, what kind of functionality is in scope for this specific library? What is for sure permanently out of scope? Here are some examples of functionality I have either finished or have stubbed out that I would like to know if I should pitch or contribute to this specific library:

  • Client-side auto-truncate messages payload to context max (currently planning to wrap useChat in a truncate handler to manage messages state prior to new appends)
  • Server-side auto-truncate messages payload to context max (currently pre-processing messages payload before .call)
  • Server-side streaming messages + completion token counting
  • SSE streaming to send token events as type "token" which enables addition non-token events like status (probably doable in a custom callback)
  • Caching messages to a database for later recall

Or is the library too new to ask about a strict scope? Or a roadmap to demonstrate the expected scope? Thank you!!

bioshazard avatar Jun 23 '23 15:06 bioshazard