ai icon indicating copy to clipboard operation
ai copied to clipboard
verified publisher icon vercel

client-side `onToolCall` with model output (making client side tool return image as result)

Open softmarshmallow opened this issue 2 months ago • 1 comments

Description

https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#multi-modal-tool-results

This explains how a tool can return a image result.

How can the same pattern be followed from client side onToolCall?

addToolOutput does not allow adding image as output. only defined / dynamic structured output is allowed.

will I need custom chain?

  1. takeClientScreenshot (client) => upload, return resource id
  2. downloadResource (server with execute defined) => fetch, add to model output.

AI SDK Version

  • "@ai-sdk/react": "3.0.0-beta.94",
  • "ai": "6.0.0-beta.94",

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

softmarshmallow avatar Nov 07 '25 15:11 softmarshmallow

Is this planned to be supported in v6?

softmarshmallow avatar Nov 11 '25 07:11 softmarshmallow