pydantic-ai
pydantic-ai copied to clipboard
pydantic
Support OpenAI web search tool events in streaming mode
Description
When using the run_stream function with an agent that has OpenAI's new Web Search tool enabled, I see a sequence of warnings printed to the console about these unhandled event types:
- ResponseWebSearchCallInProgressEvent
- ResponseWebSearchCallSearchingEvent
- ResponseWebSearchCallCompletedEvent
- ResponseTextAnnotationDeltaEvent
References
No response
Hi @adsouza, we don't yet support built in provider tools, but work to support them is underway in #1722.
If you have any more details on how you want to use OAI's web search tool, that context would be useful?
Hi @adsouza, we don't yet support built in provider tools, but work to support them is underway in #1722.
If you have any more details on how you want to use OAI's web search tool, that context would be useful?
I'm using it to let users get answers to questions about a product in the context of a product details page. I pass it the user question along with the model ID & URL of the product, using the system prompt to tell it to answer the question about the product. But it takes some time to respond to I am having it stream the reply. Is that helpful?
That's useful thanks.
Most likely the reason it's slow is that inside OAI the flow is:
- LLM: interpret the question, decide to make a tool call to web search (maybe 1-2s)
- Search tool: Search the web with that URL, get the page content, return it to the LLM (1s if you're lucky)
- LLM: interpret the question, it's response and the page content, and decide on final response (maybe 2s)
Without knowing much about your application, it might be faster if you get the page content, summarise it and send it to the LLM as part of the prompt, then there's only one LLM call.
Built in tools like search are not magic, they are easier to configure, but their only real performance improvement is saving network latency, which is generally <100ms anyway.
That's useful thanks.
Most likely the reason it's slow is that inside OAI the flow is:
- LLM: interpret the question, decide to make a tool call to web search (maybe 1-2s)
- Search tool: Search the web with that URL, get the page content, return it to the LLM (1s if you're lucky)
- LLM: interpret the question, it's response and the page content, and decide on final response (maybe 2s)
Without knowing much about your application, it might be faster if you get the page content, summarise it and send it to the LLM as part of the prompt, then there's only one LLM call.
Built in tools like search are not magic, they are easier to configure, but their only real performance improvement is saving network latency, which is generally <100ms anyway.
This is very likely true; fetching the page content is harder than it sounds though, since it's not a page we host, summarizing it would mean another LLM call anyway.
