pydantic-ai icon indicating copy to clipboard operation
pydantic-ai copied to clipboard

Correct message history order in streamed structured response with tool calls

Open siavashg opened this issue 11 months ago • 3 comments

Streamed responses with tool calls would previously order the ToolReturn message before the ModelStructuredResponse (which invoked the tool call in the first place) in the message history from agent.all_messages(). This corrects the message order.

Fixes #234

siavashg avatar Dec 12 '24 21:12 siavashg

I'm guessing this might get fixed in https://github.com/pydantic/pydantic-ai/pull/259

sydney-runkle avatar Dec 14 '24 22:12 sydney-runkle

Thanks for looking into it but it doesn't look like #259 fixes this.

Here the ToolReturnPart is returned before the ToolCallPart has even been made (per the message history!): https://github.com/pydantic/pydantic-ai/blob/dcba8cafd2f91bed7bc1b6a6ecd004fcd6994019/tests/test_streaming.py#L209-L225

And here the ToolReturnPart is ordered ahead of the ToolCallPart. https://github.com/pydantic/pydantic-ai/blob/dcba8cafd2f91bed7bc1b6a6ecd004fcd6994019/tests/test_streaming.py#L239-L257

This can easily be tested. If you play back this history against OpenAI's API it will refuse to accept it since the tool call returns are listed before the actual tool calls.

siavashg avatar Dec 16 '24 10:12 siavashg

Principle here looks good, but there's lots of conflicts and I want to change the logic a little, so I'll create a new PR to fix this.

samuelcolvin avatar Dec 16 '24 11:12 samuelcolvin

Closing in favor of #274

sydney-runkle avatar Dec 16 '24 14:12 sydney-runkle