vercel
Support OpenAI Vector Store with the new Response API
Feature Description
We need a solution to support file search when using the new response api with the AI SDK.
const productDocs = await openai.vectorStores.create({
name: "Product Documentation",
file_ids: [file1.id, file2.id, file3.id],
});
const response = await openai.responses.create({
model: "gpt-4o-mini",
tools: [{
type: "file_search",
vector_store_ids: [productDocs.id],
}],
input: "What is deep research by OpenAI?",
});
console.log(response.output_text);
News link: https://openai.com/index/new-tools-for-building-agents/
Use Cases
No response
Additional context
No response
@Pablo4Coding you can use a custom tool for now which works the same way:
tool({
description:`A file search tool`,
parameters: z.object({
query:z.string().describe("The search query. This is a search query you need to generate to assist the user. Whenever you have a question search for relevant files to enhance your context.")
}),
execute: async ({ query }) => {
try {
const results = await $fetch(`https://api.openai.com/v1/vector_stores/${agent.oai_vector_store_id}/search`, { method:"POST", body: { query }, headers: { "Authorization": `Bearer ${openai_api_key}`}, retry:5, retryDelay:1000 })
if (!results?.data?.length){
return "No files found"
} else {
return [
"<sources>",
results.data.flatMap(res => [
`<result file_id='${res.file_id}' file_name='${res.file_name}'>`,
...res.content.map(part => `<content>${part.text}</content>`),
`</result>`
]),
"</sources>"
].join('\n')
}
} catch (e) {
console.error(e)
console.error(`OPENAI File search error: ${e?.status}`)
return `Error ${e?.status}: Vector file store seems offline. Please try again in a minute.`
}
}
})
I would recommend, if you're currently using a custom performance query, to follow OpenAI's file search best practices.
OpenAI's tool performs not only multi-query searches but also ranking and other deep actions which are not explicitly disclosed.
To achieve results as close as possible to OpenAI’s method—besides properly handling the result and system prompts for optimized queries—I suggest allowing the model to generate multiple distinct queries for a single search.
Example of generated query:
// Original query: "What is the return policy?"
const queries = [
"Return policy terms",
"Refund guidelines and process",
"How to return an item",
"Exchange or return instructions",
"Time limit for product returns"
];
tool({
description:`A file search tool`,
parameters: z.object({
queries: z.array(z.string()).describe("An array of search queries. Generate multiple diverse queries to find relevant files that could assist the user.")
}),
execute: async ({ queries }) => {
try {
// Pass the entire array of queries directly to the endpoint
const results = await $fetch(`https://api.openai.com/v1/vector_stores/${agent.oai_vector_store_id}/search`, {
method: "POST",
body: { query: queries },
headers: { "Authorization": `Bearer ${openai_api_key}`},
retry: 5,
retryDelay: 1000
})
if (!results?.data?.length){
return "No files found"
} else {
return [
"<sources>",
results.data.flatMap(res => [
`<result file_id='${res.file_id}' file_name='${res.file_name}'>`,
...res.content.map(part => `<content>${part.text}</content>`),
`</result>`
]),
"</sources>"
].join('\n')
}
} catch (e) {
console.error(e)
console.error(`OPENAI File search error: ${e?.status}`)
return `Error ${e?.status}: Vector file store seems offline. Please try again in a minute.`
}
}
})
Does anyone know if there is already some work started to integrate this?
@RobSchilderr there's a WIP in #5141 , but hasn't been merged, in the meantime you could build it and patch the dependency.
updated the workaround, because the previous solutions have errorss
const fileSearch = tool({
description:`description`,
parameters: z.object({
queries: z.array(z.string()).describe("An array of search queries. Generate multiple diverse queries to find relevant files that could assist the user.")
}),
execute: async ({ queries }) => {
try {
// Pass the entire array of queries directly to the endpoint
const response = await fetch(`https://api.openai.com/v1/vector_stores/<vsid>/search`, {
method: "POST",
body: JSON.stringify({ query: queries, max_num_results: 3 }),
headers: { "Authorization": `Bearer ${process.env.OPENAI_API_KEY}`, "Content-Type": "application/json" },
})
const json = await response.json()
if (!json?.data?.length){
console.log('no data')
return ""
}
return [
json.data.flatMap((res:any) => [
...res.content.map((part:any) => `${part.text}`),
]),
]
} catch (e) {
console.error('OPENAI File search error:',e)
return ''
}
}
})
