ai
ai copied to clipboard
vercel
feat (provider/openai): support file search tool
Extending the openai-tools to support the "file_search" tool call documented by OpenAI here: https://platform.openai.com/docs/guides/tools-file-search?lang=javascript
Note: originally set this up as "file_search_preview", but opted for "file_search" because OpenAI does not have this tool tagged as a preview feature and the tool name in the OpenAI docs is "file_search".
Nice! without parsing their special outputs how will this work in practice?
When I tested it to see what would happen when adding in a vector_store_id it returned a response as expected through the "text" value with references to the vector store files (indicating to me that it worked to perform the file search).
The "text" value (returned by the new OpenAI Responses API) should work as expected when using the file_search tool. I'll work on adding some handlers for the new "annotations" value that is returned by the Responses API for better clarity.
@lgrammel I should have checked before responding. The "annotations" property is already available on the OpenAIResponsesAssistantMessage. This should be enough to pass through those annotations through the SDK response.
export type OpenAIResponsesAssistantMessage = {
role: 'assistant';
content: Array<{
type: 'output_text';
text: string;
annotations?: Array<{
type: 'file_citation';
index: number;
file_id: string;
filename: string;
} | {
type: 'url_citation';
start_index: number;
end_index: number;
url: string;
title: string;
}>;
}>;
};
This matches the OpenAI docs for the file search tool: https://platform.openai.com/docs/guides/tools-file-search
EDIT: I didn't realize I actually made this change 🤦♂️ The update to this type has been pushed to this PR.
Ok made some adjustments to ensure the annotations information is being returned properly through the sdk. Tested this locally and got a nice sources value when trying out with the file_search tool.
sources: [
{
sourceType: 'file',
id: 'PSvUjMxv2hlJ4vuj',
fileId: 'file-IhP983o4sPmYkgGOtVdureJO',
filename: 'Artificial Intelligence (AI) Policy.txt'
}
]
I've also updated the next-openai examples to account for either url or file source annotations.
Any chance we could get this release as experimental_alpha and then migrate to one which supports all providers? I would love to migrate a lot of Assistants to the new responses api but I'm blocked by the file_search limitation
@lgrammel sorry for the possible spam, but any plans to merge this soon? Any change needed to the PR?
seems like blocked, I really need this tool, I could use assistant to move this into openai could work for some of you, but it doesn't work for me as I am trying to use assistant ui cloud which doesn't support assistant.
@lgrammel do you need any help on this? It seems the responses api is not compatible with APIs from Anthropic/xAI especially these custom constructs like file_search, web_search in the responses API?
https://apimagic.ai/blog/switching-assistant-responses-api
We can either use the existing tools parameter available on the ai-sdk methods or add it to the providerOptions like below:
const providerOptions = {
openai: {
tools: [
{
type: 'file_search',
vector_store_ids: ['vs_1234567890'],
max_num_results: 20,
},
],
tool_choice: { type: 'tool', tool_name: 'file_search' },
},
};
