ControlFlow
ControlFlow copied to clipboard
Inconsistent streaming deltas
Description
I'm getting inconsistent streaming deltas when calling the same task - for example, sometimes the task produces agent-tool-call-delta and sometimes it produces agent-content-delta, seemingly randomly. This makes it hard to stream the content to an end user, since I manually have to strip JSON from the tool call deltas, which might mess up the content if it actually includes JSON.
The below code might produce (as a stream):
{"result":"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua."}
or it might produce
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Is there a better way to stream such that the stream yield is always a agent-content-delta?
Example Code
import controlflow as cf
# Stream all events
for event, snapshot, delta in cf.run("Give me the first paragraph of lorum Epsom", stream=True, handlers=[]):
if event.event == "agent-content-delta":
print(delta, end="")
elif event.event == "agent-tool-call-delta":
print(delta['args'],end="")
Version Information
ControlFlow version: 0.12.1
Prefect version: 3.2.6
LangChain Core version: 0.3.37
Python version: 3.11.11
Platform: macOS-15.3-arm64-arm-64bit
Using gpt-4o-mini on Azure.
Additional Context
No response