big-AGI icon indicating copy to clipboard operation
big-AGI copied to clipboard

Function streaming

Open ramicaza opened this issue 9 months ago • 5 comments

Function calling is extremely useful but it's a little tricky to implement it together with message streaming because of the weird way functions are streamed by openai. By streaming both messages and functions, it allows for fluid UX like the one displayed below. Notice how the model goes from streaming a response, right to calling the function it needs.

This PR is to enable function call streaming such that modules like ReAct and other future ones can have fluid UIs like the one below.

https://github.com/enricoros/big-agi/assets/8342756/43b9d352-84e0-4a75-8e22-667b43d567a8

ramicaza avatar Nov 19 '23 18:11 ramicaza