Allow showing a spinner while client-side tools are running

[d3lm]

Feature Description

There is currently no out-of-the-box way to show a spinner while client-side tools are running / in progress. Consider the following client side code:

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, addToolResult } =
    useChat({
      maxToolRoundtrips: 5,
      
      async onToolCall({ toolCall }) {
        // do something that takes a while
        await new Promise((resolve) => setTimeout(resolve, 5000));
        
        return 'Success';
      },
    });

  return (
    <div>
      {messages?.map((m: Message) => (
        <div key={m.id}>
          <strong>{m.role}:</strong>
          {m.content}
          {m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
            
            // we never see the invoked client-side tool here until it has a result
            
            // for example the following doesn't work
            
            if (!('result' in toolInvocation)) {
              return <div>Tool running...</div>
            }
          })}
          <br />
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}

Use Case

As you can see in the provided code, the use case is to show a spinner while a client-side tool is running and potentially show the result once the tool is done. Maybe we could add a status property that can be either 'running' | 'complete'. I think we can not always rely on the result property because it could be undefined if you don't pass something back to the model.

The only case that works and allows us to show a spinner is when a server-side tool is executed. In that case the tool invocation is added to the message and doesn't have a result. On the server side one could use StreamData and pass some status to be handled on the client.

Additional context

No response

[BrianHung]

@d3lm Should be possible now with streaming tool calls: #2295 tool.state === 'partial-call'.

[lgrammel]

@BrianHung this is is about client-side tool calling, and how we await the results from what I understand. It would require switching to parallel invocation and waiting, but that can lead to race conditions so it requires careful testing.

[BrianHung]

Oh, its not about streaming tool calls, its about the tool call being called and waiting for it to finish. Isn't that a scenario here with askForConfirmation where the args is rendered, but no result yet.

https://github.com/vercel/ai/blob/ae3366323f6a94a80371d918a0a9347310eb35a3/examples/next-openai/app/use-chat-tools/page.tsx#L48-L78

[lgrammel]

@BrianHung what you mention is an interactive client-side tool. for that it works. there are also client side tools that run automatically in "onToolCall". For those it's currently not possibly to show the spinner.