plz.el icon indicating copy to clipboard operation
plz.el copied to clipboard

Request for streaming support

Open ahyatt opened this issue 1 year ago • 4 comments

This seems like a great library that I would love to use in llm (also on GNU ELPA). However, it would first need to support streaming, taking in a function that gets called with... well, it could depend. Right now the way it works in llm is that the user function gets called with the complete text of the response so far. But probably calling with new data would work as well.

The streaming callback is in addition to the "then" callback, which still needs to exist (clients probably will want to know when the task is done).

So a possible plz request from the client side with streaming might look like (plz post "http://llm-provider/model :body (json-encode request-data) :then final-callback :during streaming-callback).

ahyatt avatar Nov 01 '23 01:11 ahyatt