Error responses from an API are parsed as Text in `useChat` React hook

[jonsherrard]

Description

I have an API handler based on the docs that looks like this.

import OpenAI from "openai";
import { OpenAIStream, StreamingTextResponse } from "ai";
import { initialMessage } from "./initial-message";
import { NextResponse } from "next/server";

const openai = new OpenAI({
  apiKey: process.env["OPENAI_API_KEY"],
});

export const runtime = "experimental-edge";

export default async function (req: any) {
  const im = await initialMessage();
  const { messages } = await req.json();
  try {
    const response = await openai.chat.completions.create({
      model: "gpt-4-1106-preview",
      stream: true,
      messages: [{ role: "assistant", content: im }, ...messages],
    });
    const stream = OpenAIStream(response);
    return new StreamingTextResponse(stream);
  } catch (error) {
    console.log("Caught Open AI error");
    // Check if the error is an APIError
    if (error instanceof OpenAI.APIError) {
      console.log("Open AI API Error");
      const { name, status, headers, message } = error;
      return NextResponse.json({ name, status, headers, message }, { status });
    } else {
      throw error;
    }
  }
}

when using const {error} = useChat(), the error is a string of the object response from the server.

Due to line: https://github.com/vercel/ai/blob/e250e16806a856c186f650825b46a5af8f09bcf1/packages/core/react/use-chat.ts#L191 using res.text() I think.

To get around it in my code I have to JSON.parse(errror.message)?.message, which is a bit odd.

Code example

import OpenAI from "openai";
import { OpenAIStream, StreamingTextResponse } from "ai";
import { initialMessage } from "./initial-message";
import { NextResponse } from "next/server";

const openai = new OpenAI({
  apiKey: process.env["OPENAI_API_KEY"],
});

export const runtime = "experimental-edge";

export default async function (req: any) {
  const im = await initialMessage();
  const { messages } = await req.json();
  try {
    const response = await openai.chat.completions.create({
      model: "gpt-4-1106-preview",
      stream: true,
      messages: [{ role: "assistant", content: im }, ...messages],
    });
    const stream = OpenAIStream(response);
    return new StreamingTextResponse(stream);
  } catch (error) {
    console.log("Caught Open AI error");
    // Check if the error is an APIError
    if (error instanceof OpenAI.APIError) {
      console.log("Open AI API Error");
      const { name, status, headers, message } = error;
      return NextResponse.json({ name, status, headers, message }, { status });
    } else {
      throw error;
    }
  }
}

Additional context

No response

[tgonzales]

Hi @jonsherrard const im = await initialMessage() Do you make any calls to the database to fetch this message? or something like that? It is advisable to validate whether this initialMessage is returning successfully if(!im) throw new Error

[jonsherrard]

@tgonzales Thanks it's just a long string :), it did make a fetch at one point, hence the await.. I can refactor it to just a straight import.

[tgonzales]

I understand, the error indicates there is something wrong with the messages, do a partial debugger to see if the format of the messages is correct.

[lgrammel]

I've tried to reproduce the error with the following example:

// route.ts
import { NextResponse } from 'next/server';

// IMPORTANT! Set the runtime to edge
export const runtime = 'edge';

export async function POST(req: Request) {
  try {
    throw 'Not implemented';
  } catch (error) {
    return NextResponse.json(
      { name: '403 forbidden', status: 403, message: 'Test' },
      { status: 403 },
    );
  }
}

// page.tsx
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, error } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {error ? <div className="mb-8 text-red-500">{error.message}</div> : null}

      {messages.length > 0
        ? messages.map(m => (
            <div key={m.id} className="whitespace-pre-wrap">
              {m.role === 'user' ? 'User: ' : 'AI: '}
              {m.content}
            </div>
          ))
        : null}

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}

It correctly displayed the error:

Can you give me more information about your code and how you fixed the issue? I did not find any difference in your examples. From what I can tell, it is not necessarily related to useChat.