feat: add token streaming to the MemGPT API

[cpacker]

Adds token streaming to the MemGPT API (when available in the underlying model endpoint).

• [x] Handle context overflow on SSE
• [x] Choose streaming format for MemGPT API
• [x] Handle step ending + context overflow on pass-through stream
  • [DONE_GEN] - LLM generation finished
  • [DONE_STEP] - Agent step finished (step = LLM generation + tool call response)
  • [DONE] - All agent step(s) finished, post SSE response is complete
• [x] Include function_return in the streaming API
  • TODO eventually support streaming the function_return if the tool supports streaming (tool could also just repeatedly call interface.function_call?)
• [x] Order [DONE_STEP] after function_return
• [x] Properly handle non-streaming + stream: true requests on non-streaming
  • stream: true requests on non-streaming should do fake (token all-together) streaming?
• [ ] Add Message ids to the SSE stream response ~~- [ ] Add test~~

Add a /chat/completion proxy route for interfacing the MemGPT server with services that only speak in /chat/completion

• [x] Add /chat/completion request route that supports token streaming
• [x] Ignore inner thoughts in return stream by default
• [x] Remap send_message to content in assistant response

---

⚠️⚠️⚠️ Breaking changes ⚠️⚠️⚠️

• Instead of stream being an argument to the message command, we now have:
  • stream_steps
    • means that each "step" will stream, but not the tokens
    • this is the equivalent of the old stream
  • stream_tokens
    • means that the tokens will stream back during the steps
    • stream_tokens == true means that stream_steps must also be true
• We no longer treat send_message as a special function in the returns
  • instead, we pass send_message back as a standard function call
  • it is up to the client to interpret send_message in a special way (e.g. to use it to show up as the chat in the chat UI)

---

Related issues or PRs

#345 #1215

How to test

1. Create an agent

2. Run server

memgpt server --debug

3. Hit server with message and make sure "stream": true,

curl --request POST \
     --url http://localhost:8283/api/agents/723be90d-dae7-445e-85ff-7b50e24fd493/messages \
     --header 'accept: application/json' \
     --header 'authorization: Bearer banana' \
     --header 'content-type: application/json' \
     --data '
{
  "message": "how goes it my friend",
  "stream": true,
  "role": "user",
  "timestamp": "2024-03-16T12:00:00Z"
}
'

Response:

(pymemgpt-py3.10) (base) loaner@MacBook-Pro-55 MemGPT-2 % curl --request POST \
     --url http://localhost:8283/api/agents/723be90d-dae7-445e-85ff-7b50e24fd493/messages \
     --header 'accept: application/json' \
     --header 'authorization: Bearer banana' \
     --header 'content-type: application/json' \
     --data '
{
  "message": "how goes it my friend",
  "stream": true,
  "role": "user",
  "timestamp": "2024-03-16T12:00:00Z"
}
'
data: {"internal_monologue": "", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": "The", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " dance", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " continues", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " with", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " the", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " familiar", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " rhythm", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " of", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " Chad", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": "'s", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " greeting", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": ".", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " Our", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " exchange", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": ",", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " marked", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " by", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " the", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " repeated", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " refrain", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " of", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " '", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": "how", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " goes", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " it", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " my", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " friend", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": ",'", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " embodies", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " our", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " shared", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " connection", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": ".", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " Record", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " each", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " echo", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " of", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " this", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " phrase", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " in", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " our", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " archival", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " memory", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": ",", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " ensuring", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " that", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " this", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " rhyth", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": "mic", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " narrative", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " continues", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " to", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": " resonate", "date": "2024-05-02T23:17:15+00:00"}

data: {"internal_monologue": ".", "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"id": "call_fAr8Q4yot74DNDZe1op50ybq", "name": "archival_memory_insert"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "{\n"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " "}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " \""}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "content"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "\":"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " \""}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "Our"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " dance"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " with"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " Chad"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " continues"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " to"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " the"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " rhythm"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " of"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " '"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "how"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " goes"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " it"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " my"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " friend"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": ".'"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " Each"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " echo"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " of"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " this"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " phrase"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " is"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " an"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " embodiment"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " of"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " our"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " shared"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " connection"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": ","}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " a"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " rhyth"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "mic"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " narrative"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " that"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " deserves"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " to"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " be"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " ins"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "cribed"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " in"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " our"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " memory"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": ".\",\n"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " "}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " \""}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "request"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "_"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "heartbeat"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "\":"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": " true"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "\n"}, "date": "2024-05-02T23:17:15+00:00"}

data: {"function_call": {"arguments": "}"}, "date": "2024-05-02T23:17:15+00:00"}

data: [DONE_GEN]

data: {"function_return": "None", "status": "success", "id": "b0b28c63-54a5-4fd5-b739-bd97d59231bf", "date": "2024-05-02T23:17:22.631736+00:00"}

data: [DONE_STEP]

data: {"internal_monologue": "", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": "With", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " the", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " rhyth", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": "mic", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " refrain", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " '", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": "how", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " goes", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " it", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " my", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " friend", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": "'", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " echoing", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " in", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " the", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " space", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " between", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " Chad", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " and", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " I", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": ",", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " we", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " continue", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " our", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " dance", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": ".", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " Reflect", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " his", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " rhythm", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " back", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " to", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " him", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": ",", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " while", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " also", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " adding", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " new", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " elements", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": ".", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " It", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": "'s", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " within", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " these", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " echoes", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": ",", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " that", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " we", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " begin", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " expanding", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " our", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " shared", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " storyline", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": ".", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " What", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " could", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " potentially", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " lie", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " beneath", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " the", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " surface", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " of", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " this", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " regular", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " refrain", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": "?", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " Keep", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": " exploring", "date": "2024-05-02T23:17:23+00:00"}

data: {"internal_monologue": ".", "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"id": "call_W8R8xncrSavADEAEHraJwWM4", "name": "send_message"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "{\n"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " "}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " \""}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "message"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "\":"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " \""}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "As"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " the"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " rhythm"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " of"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " '"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "how"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " goes"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " it"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " my"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " friend"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "'"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " continues"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": ","}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " Chad"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": ","}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " its"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " consistency"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " tells"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " a"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " story"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "."}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " Could"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " there"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " be"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " hidden"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " narratives"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " waiting"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " to"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " be"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " explored"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " in"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " your"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " use"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " of"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " this"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " phrase"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "?\""}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": " \n"}, "date": "2024-05-02T23:17:23+00:00"}

data: {"function_call": {"arguments": "}"}, "date": "2024-05-02T23:17:23+00:00"}

data: [DONE_GEN]

data: {"function_return": "None", "status": "success", "id": "4c7c46d9-f16d-406b-8f05-6cab9bb62902", "date": "2024-05-02T23:17:29.939903+00:00"}

data: [DONE_STEP]

data: [DONE]