ai

ai copied to clipboard

I'm using the following code:

const {  isLoading, messages, input, handleInputChange, handleSubmit } = useChat()  

...

<fieldset disabled={isLoading}>
                <textarea
                  value={input}
                  onChange={handleInputChange}
                  placeholder="Send a message..."
                />
                <button className="btn" type="submit">
                  Send
                </button>
              </fieldset>

When a stream starts, isLoading is never resets to false. This is obviously a bug.

Which framework are you using?

Next.js

Can you provide your entire code? It may be a bug with handleSubmit if that's what you're using for the form, but I can't reproduce with some quick tests.

'use client'

import { Message } from 'ai/react'
import { useChat } from 'ai/react'
import { ChatRequest, FunctionCallHandler, nanoid } from 'ai'

export default function Chat() {
  const functionCallHandler: FunctionCallHandler = async (
    chatMessages,
    functionCall
  ) => {
    if (functionCall.name === 'eval_code_in_browser') {
      if (functionCall.arguments) {
        // Parsing here does not always work since it seems that some characters in generated code aren't escaped properly.
        const parsedFunctionCallArguments: { code: string } = JSON.parse(
          functionCall.arguments
        )
        // WARNING: Do NOT do this in real-world applications!
        eval(parsedFunctionCallArguments.code)
        const functionResponse = {
          messages: [
            ...chatMessages,
            {
              id: nanoid(),
              name: 'eval_code_in_browser',
              role: 'function' as const,
              content: parsedFunctionCallArguments.code
            }
          ]
        }
        return functionResponse
      }
    }
  }

  const { isLoading, messages, input, handleInputChange, handleSubmit } =
    useChat()

  // Generate a map of message role to text color
  const roleToColorMap: Record<Message['role'], string> = {
    system: 'red',
    user: 'black',
    function: 'blue',
    assistant: 'green'
  }

  return (
    <form onSubmit={handleSubmit}>
      <fieldset disabled={isLoading}>
        <textarea
          value={input}
          onChange={handleInputChange}
          placeholder="Send a message..."
        />
        <button className="btn" type="submit">
          Send
        </button>
      </fieldset>
      <div className="messages">
        {messages.map(message => (
          <div key={message.id} style={{ color: roleToColorMap[message.role] }}>
            {message.content}
          </div>
        ))}
      </div>
    </form>
  )
}

works fine for me

@MaxLeiter I have the same issue as @blazestudios23 on my system. I tried your code and the problem remains the same. I used 2.2.9 and 2.2.10 of the package.

In the route.ts, I use langchains ChatOllama with a StringOutputParser. I currently have no way to test with ChatOpenAI and the BytesOutputParser to see if it makes a difference.

Mine actually does go back to true, but it takes about 30 seconds after the chat response ends. So some kind of issue with it not registering that the chat is done.

here's my page:

export default function ChatPage({
  chatId,
  title  = "AI Bot",
  messages,
}: InferGetServerSidePropsType<typeof getServerSideProps>) {
  const [newChatId, setNewChatId] = useState<string | null>(null);
  const [originalChatId, setOriginalChatId] = useState(chatId);
  const [showSidebar, setShowSidebar] = useState(false);

  const routeHasChanged = chatId !== originalChatId;

  if(!messages.length){
    messages = [{
      id: "start",
      content: "Hello! Ask me anything.",
      role: RoleEnum.assistant
    }]
  }

  const { isLoading, messages: messagesStream, input, handleInputChange, handleSubmit } = useChat({
    id: chatId,
    initialMessages: messages
  })  
  
  const router = useRouter();

  // when our route changes
  useEffect(() => {
    setNewChatId(null);
  }, [chatId]);

  // if we create new chat then navigate to it
  useEffect(() => {
    if (!isLoading && newChatId) {
      setNewChatId(null);
      router.push(`/chat/${newChatId}`);
    }
  }, [newChatId, isLoading, router]);

  return (
    <>
      <Head>
        <title>{title}</title>
      </Head>
      <div className={`grid h-screen ${showSidebar ? "grid-cols-[260px_1fr]" : ""} relative`}>
      {showSidebar ? (
        <ChatSidebar setShowSidebar={setShowSidebar} chatId={chatId} />
      ) : (<>
            <FontAwesomeIcon onClick={() => setShowSidebar(!showSidebar)} color="white" icon={faArrowAltCircleRight} className='absolute left-1 top-1' />
            
          </>)}
        <div className="flex flex-col overflow-hidden bg-gray-700 ">
          <div className="flex-1 flex flex-col-reverse overflow-y-auto text-white">
            {!messagesStream.length && (
              <div className="m-auto flex items-center justify-center text-center">
                <div>
                  <FontAwesomeIcon
                    icon={faRobot}
                    className="text-6xl text-emerald-200"
                  />
                  <h1 className="mt-2 text-4xl font-bold text-white/50 ">
                    Ask me a question!
                  </h1>
                </div>
              </div>
            )}
            {!!messagesStream.length && (
              <div className="mb-auto">
                {messagesStream.map((message) => (
                  <Message
                    role={message.role}
                    id={message.id}
                    key={message.id}
                    content={message.content}
                  />
                ))}
              </div>
            )}
          </div>
          <footer className="bg-gray-800 p-10">
            <form onSubmit={(e) => handleSubmit(e, {
              options:{
                body: {
                  chatId
                }
              }
            })}>
              <fieldset className="flex gap-2" disabled={isLoading}>
                <textarea
                  value={input}
                  onChange={handleInputChange}
                  className="w-full resize-none rounded-md bg-gray-700 p-2 text-white focus:border-emerald-500 focus:bg-gray-600 focus:outline focus:outline-emerald-500"
                  placeholder="Send a message..."
                />
                <button className="btn" type="submit">
                  Send
                </button>
              </fieldset>
            </form>
          </footer>
        </div>
      </div>
    </>
  );
}

I experience the same issue with useCompletion after awaiting complete(prompt). isLoading stayed true whilst the stream already ended. I found that using the BytesOutputParser for Langchain resolved the issue. The docs state:

/**
  * Chat models stream message chunks rather than bytes, so this
  * output parser handles serialization and encoding.
  */
const outputParser = new BytesOutputParser();

Experiencing something similar here when the stream is interrupted before it starts sending data.

1. Start a completion
2. Switch to a different chat id before it sends data through
3. The hook gets stuck with isLoading set to true for that chat id

Experiencing same issue with Vue. Anyone got it resolved?