ai

ai copied to clipboard

Currently options are fixed. However I need a way to change the body. My code looks like this (with vue.js but I think the idea is the same with react or any other library):

<template>
	<div>
	  <button @click="n++; input = Math.random(); ">Increase to {{n+1}}</button>
	  <br /> <br/>
	  <button @click="handleSubmit">Submit</button>
	</div>
</template>

<script setup>
import { useChat } from 'ai/vue'

onMounted(() => input.value = Math.random()) // Otherwise req is not sent

const n = ref(0)
	const optio = computed(() => {

	})
	const { handleSubmit, input } = useChat({
		api:"/404",
		body: { n: n.value },
		onError: (e => alert("Callback works"))
	})
</script>

The body will allways be 0 (the inital value) no matter what n is. Ideally body would accept a function, like getBody or we would have a way to update the useChat options.

Can you not create a getBody() function or some equivalent?

useChat({ 
    body: getBody()
}) 

I tried, it doesn't work, it gets called on creation and never again.

I tried with:

	const n = ref(0)
	const getBody = () => {
		console.log("Getting body")
		return { n: n.value }
	}
	const { handleSubmit, input, data, reload} = useChat({
		api:"/404",
		body: getBody(),
		onError: (e => alert("Callback works"))
	})
	

Ah yeah I was rushing, sorry -- body should be changed to accept a function. Would you like to submit a PR?

Would you like to submit a PR?

I really wouldn't know from where to start! But I agree a function for body would be great.

In each of the use-chat files (so for react, solid, vue, svelte) there's this line https://github.com/vercel/ai/blob/d6b8948dafdd06eb170ac4cdd8b72d3e8f7f410f/packages/core/vue/use-chat.ts#L123C16-L123C16

          ...body,
          ...options?.body

If you adjust the types of body (in shared/types.ts) to also accept a () => Request.BodyInit these lines can become

	...(body instanceof Function ? body() : body)

Ok I'll give it a try

@MaxLeiter currently there are 2 bodies, options.body and body. I should replace both right? Like this:

    //   ...extraMetadata.body, // Remove this
    //   ...chatRequest.options?.body, // Remove this
	  ...(extraMetadata?.body instanceof Function ? extraMetadata.body() : extraMetadata?.body),
	  ...(chatRequest?.options?.body instanceof Function ? chatRequest.options.body() : chatRequest?.options?.body),

Let me know: https://github.com/vercel/ai/pull/477

Not sure if this is a bug but when you do n++ the internal ref of body should be updated. That’s how it works in the React version if n is a state, and the Vue version should do the same.

This discussion in this thread then explains why I'm having the same issue ? https://github.com/vercel/ai/discussions/508

@cosbgn did you ever resolve your issue? I'm also using vue (Nuxt).

@dosstx hi, I ended up using a custom solution (without vercel/ai).

This is how I do it with nuxt:

From the /api/chat return sendStream it's built in as it's part of h3. Then on the client do something like this:

1. use fetch('/api/chat') as you need the full request body (i.e. not parsed by $fetch).

Then on the client do:

		const reader = resp.body.getReader();
		const decoder = new TextDecoder("utf-8") 
		while (true) {
               	const { value, done } = await reader.read()
				if (done) { 
					break;  // Break while loop
				}
				const lines = decoder.decode(value).toString().split('\n').filter(line => line.trim() !== '')
				for (const line of lines){
					const message = line.replace(/^data: /, '')
					if (message === '[DONE]') { return } // Stream finished
					full_json += message // In case there was something before like a non ready json
					if (full_json.endsWith("}]}")) {
                                               // Sometimes a line is not a full-json yet
						const parsed = JSON.parse(full_json)
                                                full_json = '' // Reset because it was parsed successfully
						content = content + (parsed?.choices?.[0]?.delta?.content ?? '')
				}
               }

I believe I may have fixed this issue for my case. I submitted a PR here: https://github.com/vercel/ai/pull/511

In Vue 3, refs should be unwrapped inside the composable to make them reactive. By using the ref pattern, you allow the composable to react to changes in the value. The missing piece here is that the useChat utility should use unref(body) instead of directly accessing the .value property.

@cosbgn I'll check out your suggestion, thanks!

Currently I do this using a state variable.

const {
    messages,
    setMessages,
    input,
    reload,
    stop,
    isLoading,
    handleInputChange,
    handleSubmit,
  } = useChat({
    api: "/api/chat",
    body: {
      ...state.model_config,
    },
  });

While we wait for that to be merged, here's a workaround for us Svelte folks:

	let chat = useChat({ body: { options } });
	let { input, handleSubmit, messages, isLoading } = chat;
	$: {
	     chat = useChat({ body: { options } });
	     input = chat.input;
	     handleSubmit = chat.handleSubmit;
	     messages = chat.messages;
	     isLoading = chat.isLoading;
	   }

Hi, I wonder if this problem has been completely solved? Same problem here, useChat is used in vue code but the body parameter is not updated, Or how to change the code?

const chatId = ref('')
const { messages, input, handleSubmit, isLoading, stop, reload } = useChat({
  initialMessages,
  body: {
    id: chatId.value,
  },
})

@cosbgn this is possible using the ChatRequestOptions. The docs currently are inconsistent with the api (I have an issue to fix mentioned above, but I wanted to let you know incase other people come across this issue.

Docs:

      await chat.handleSubmit(e, {
        options: {
          body: {
            ...
          },
          headers: {
           ...
          },
        },

For React, you can now fully control the body that is sent to the server: https://sdk.vercel.ai/examples/next-pages/chat/use-chat-custom-body

For anyone still facing this issue. I found another way:

form calls my onSubmit which calls VercelAI handleSubmit

const { handleSubmit } = useChat({});

const onSubmit = (e: any, chatRequestOptions?: ChatRequestOptions) => {
    handleSubmit(e, {
        ...chatRequestOptions,
        options: {
            ...chatRequestOptions?.options,
            body: {
                ...chatRequestOptions?.options?.body,
                metadata: {
                    // your additional props here
                },
            },
        },
    });
};

<form on:submit={onSubmit}>
  <!-- chat logic -->
</form>