ai icon indicating copy to clipboard operation
ai copied to clipboard

Published 20 hours ago • verified publisher icon vercel

feat (rsc): multiple tool calls for streamUI

Open gclark-eightfold opened this issue 1 year ago • 7 comments

Summary

Adds the ability to invoke multiple tool calls for streamUI.

Addresses https://github.com/vercel/ai/issues/1894, https://github.com/vercel/ai/issues/1532#issuecomment-2122232969

NOTE: For usage with aiState, it's important to call aiState.done(...) only in the onFinish callback (if expecting multiple tool calls). -- Not sure if this is a breaking change.

Tasks

  • [x] rebase onto https://github.com/vercel/ai/pull/1920 once merged to pickup refactored onFinish callback changes
  • [x] implementation
  • [ ] test
  • [ ] example
  • [ ] docs
    • [ ] reference
    • [ ] example
  • [ ] changeset

gclark-eightfold avatar Jun 10 '24 20:06 gclark-eightfold

https://github.com/vercel/ai/issues/1894

bneigher avatar Jun 12 '24 03:06 bneigher

@lgrammel LMK if you have any thoughts on this implementation. You had mentioned previously that streamUI would be refactored eventually to match streamText / other patterns, so let me know if you think supporting this would be best to do after that change or before.

I've ad-hoc tested this in my own application and it does seem to work, though I've made a few comments on the implementation where things were maybe not as ideal.

gclark-eightfold avatar Jun 20 '24 18:06 gclark-eightfold

Hi Guys! Any news on releasing this feature.

dutsik-p avatar Jul 02 '24 11:07 dutsik-p

Hi guys any news on , when will this feature be released?

syedgaian avatar Jul 23 '24 05:07 syedgaian

Hi guys any news on , when will this feature be released?

Haven't put this PR up for review because it's a bit buggy after updating it to the latest AI SDK version. Currently getting these errors in my application I've patched with the changes in this PR:

  • Unexpected Fiber popped.
  • Detected multiple renderers concurrently rendering the same context provider. This is currently unsupported.
  • TypeError: Cannot read properties of null (reading 'children')

gclark-eightfold avatar Jul 23 '24 22:07 gclark-eightfold

@lgrammel I reverted https://github.com/vercel/ai/pull/2068 locally and tested on my application code and it fixed the issues described above.

Any idea how best to resolve this issue?

gclark-eightfold avatar Jul 23 '24 22:07 gclark-eightfold

Ah, I was wondering why my app kept hanging up. This would be awesome to get released

artokun avatar Aug 04 '24 05:08 artokun

I had the same requirement and ended up building a layer on top of streamText to do it. It sounds like this will be natively supported in the SDK at some point, but for the time being I think it's still something one needs to put together oneself.

Check out https://github.com/edspencer/stream-multi if you are interested - the little video there shows the ability to have the LLM respond with both streaming text and any number of tools calls.

I published it as a npm module called ai-stream-multi. It basically has the same API as streamText(), but adds the concept of Segments, which are either a Tool or Text, and of which there can be any number in a given response from the LLM. Use the onSegment callback to update the AIState.

Demo

edspencer avatar Aug 12 '24 17:08 edspencer

link all the related discussion, a new implementation could fix all these problems together. https://github.com/vercel/ai/issues/2915 https://github.com/vercel/ai/issues/1894 https://github.com/vercel/ai/issues/1895 https://github.com/vercel/ai/issues/1976 https://github.com/vercel/ai/issues/2915

zzh8829 avatar Oct 10 '24 21:10 zzh8829

I created a similar version here but using the new streamText api as the plumbing. https://github.com/vercel/ai/issues/1895#issuecomment-2405975142

zzh8829 avatar Oct 10 '24 21:10 zzh8829