Support useChat Without an API Route (Client-side usage)

[The-Best-Codes]

Feature Description

Right now, the Vercel AI SDK's useChat hook is good for managing chat state, but it requires an API route (e.g., app/api/chat/route.ts). So even in simple projects or static sites, a backend is necessary just to proxy requests. I would like to be able to use the useChat hook right on the client with a user-provided API key, with no need for an API route.

Use Cases

I can think of a lot of use cases:

• Static site generators (Next.js static export, Astro, etc.)
• No-API web-to-desktop tools, e.g., Tauri
• Browser extensions
• Vite projects

Additional context

No response

[The-Best-Codes]

Feel free to assign me! I've got a solution I'm using locally (for the React useChat hook). Let me know how you guys would implement this. 😊

[chewong]

For anyone who is looking for a pure-frontend hacky solution (I hosted a Llama 3 model with vLLM at port 5000)

const model = createOpenAICompatible({
  name: "model",
  baseURL: "http://localhost:5000/v1/"
})
const customFetch = async (url, options) => {
  const m = JSON.parse(options.body) as any;
  console.log(m)
  const result = await streamText({
    model: model.chatModel('meta-llama/Llama-3.1-8B-Instruct'),
    messages: m.messages,
    abortSignal: options.signal,
  });
  return result.toDataStreamResponse()
}

const { messages, input, handleInputChange, handleSubmit } = useChat({
  fetch: customFetch,
});

[The-Best-Codes]

@chewong This looks great, I will try it 👍

[xhiroga]

Thank you for the issue and PR. This feature is very necessary. When creating application extensions, for example, we often need to attach a web server to an existing process. In such cases, in order to use the AI SDK, it must be available from the hosted application in a statically exported state.

[lgrammel]

ai sdk 5 supports custom chat transports, which enables client only usage

[The-Best-Codes]

Awesome!! ❤ Should my PR be closed too? https://github.com/vercel/ai/pull/5207

[The-Best-Codes]

@lgrammel I've been experimenting with AI SDK v5 beta and can't really see how it will let me use a model without a backend. Is there any example in the docs?

[The-Best-Codes]

Thanks to the code example from @chewong and some research, I was able to create a client-side only (no API backend) minimal chat example. In my example, I'm using @ai-sdk/google. If anyone needs it, here it is!

// Vite 7.0.4 + React 19.1.0 + AI SDK 2.0.0-beta.11
import { createGoogleGenerativeAI } from "@ai-sdk/google"; // 2.0.0-beta.11
import { useChat } from "@ai-sdk/react"; // 2.0.0-beta.20
import { convertToModelMessages, DefaultChatTransport, streamText } from "ai"; // 5.0.0-beta.20
import { useState } from "react"; // 19.1.0

const google = createGoogleGenerativeAI({
  apiKey: import.meta.env.VITE_GOOGLE_GENERATIVE_AI_API_KEY, // Make .env with this value
});

const customFetch = async (_input: RequestInfo | URL, init?: RequestInit) => {
  const m = JSON.parse(init?.body as string);
  const result = streamText({
    model: google("gemini-2.0-flash"),
    messages: convertToModelMessages(m.messages),
    abortSignal: init?.signal as AbortSignal | undefined,
  });
  return result.toUIMessageStreamResponse();
};

function App() {
  const [input, setInput] = useState("");

  const { error, messages, sendMessage } = useChat({
    transport: new DefaultChatTransport({
      fetch: customFetch,
    }),
  });

  return (
    <main>
      {messages.map((message) => (
        <div key={message.id}>
          {message.role === "user" ? "User: " : "AI: "}
          {message.parts.map((part, i) => {
            switch (part.type) {
              case "text":
                return <div key={`${message.id}-${i}`}>{part.text}</div>;
            }
          })}
        </div>
      ))}

      <form
        onSubmit={(e) => {
          e.preventDefault();
          sendMessage({ text: input });
          setInput("");
        }}
      >
        {error && <div>Error: {error.message}</div>}
        <input
          value={input}
          placeholder="Say something..."
          onChange={(e) => setInput(e.currentTarget.value)}
        />
      </form>
    </main>
  );
}

export default App;

[lgrammel]

The recommended approach would be to implement your own chat transport, then fetch is never called.

https://github.com/vercel/ai/blob/main/packages/ai/src/ui/chat-transport.ts

[GHuyHuynh]

@The-Best-Codes your code works really well. Thank you so much for sharing!

[MichaelYuhe]

The recommended approach would be to implement your own chat transport, then fetch is never called.

https://github.com/vercel/ai/blob/main/packages/ai/src/ui/chat-transport.ts

Could you please let me know if there will be some examples available for this? I tried it out, but it does not seem to be working as expected too. https://github.com/vercel/ai/issues/5140#issuecomment-3079216487

[The-Best-Codes]

@MichaelYuhe I actually have a working demo of that too but haven't got around to sharing it. Still would love to see an example from @lgrammel as he's the pro and will know the best practices 😄

[MichaelYuhe]

@MichaelYuhe I actually have a working demo of that too but haven't got around to sharing it. Still would love to see an example from @lgrammel as he's the pro and will know the best practices 😄

I hacked my own walkthrough today. My situation may be more complex, as I am using a custom endpoint proxy server to forward requests to the actual target path.

[GHuyHuynh]

@MichaelYuhe I actually have a working demo of that too but haven't got around to sharing it. Still would love to see an example from @lgrammel as he's the pro and will know the best practices 😄

I hacked my own walkthrough today. My situation may be more complex, as I am using a custom endpoint proxy server to forward requests to the actual target path.

@MichaelYuhe did you made the proxy server with a backend server? I did thought about that too but my project constraint would not allow for a backend server.

[lhr0909]

In Electron, the proper way seems to be implementing ChatTransport interface with MessagePort so eventually fetch is not needed. The added benefit would be that the stream will be fetching from the main process and the developer console will not show the network traffic record compared to the fetch method. Keeps the prompts out of client side as well.

I am not sure if I would implement it now if the fetch method works, since in Electron it is mostly BYOK model anyway and we might not need to hide the implementation detail. If it doesn't take a long time I will post back with what I can put together in a few days.

[The-Best-Codes]

Hi all! I made a simple demo of using a custom chat transport so that @ai-sdk/react can be used in a frontend Vite application. The demo is public here:

https://github.com/The-Best-Codes/ai-sdk-vite-demo

I'd love to get Lars's feedback on this to make sure I did everything correctly 👀 Also tagging some people this might help: @lhr0909, @xhiroga, @MichaelYuhe

[lborgman]

Where can we follow AI SDK 5 and the use of it in the web browser?

[Vaib215]

How to do this in ai sdk v5 in expo

[codemyhappy]

@The-Best-Codes Good job! I'll follow you!