ControlFlow icon indicating copy to clipboard operation
ControlFlow copied to clipboard

Inconsistent streaming deltas

Open Raphtor opened this issue 10 months ago • 0 comments

Description

I'm getting inconsistent streaming deltas when calling the same task - for example, sometimes the task produces agent-tool-call-delta and sometimes it produces agent-content-delta, seemingly randomly. This makes it hard to stream the content to an end user, since I manually have to strip JSON from the tool call deltas, which might mess up the content if it actually includes JSON.

The below code might produce (as a stream):

{"result":"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua."}

or it might produce

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Is there a better way to stream such that the stream yield is always a agent-content-delta?

Example Code

import controlflow as cf
# Stream all events
for event, snapshot, delta in cf.run("Give me the first paragraph of lorum Epsom", stream=True, handlers=[]):
    if event.event == "agent-content-delta":
        print(delta, end="")
    elif event.event == "agent-tool-call-delta":
        print(delta['args'],end="")

Version Information

ControlFlow version: 0.12.1                                              
       Prefect version: 3.2.6                                               
LangChain Core version: 0.3.37                                              
        Python version: 3.11.11                                             
              Platform: macOS-15.3-arm64-arm-64bit

Using gpt-4o-mini on Azure.

Additional Context

No response

Raphtor avatar Feb 24 '25 15:02 Raphtor