pipecat icon indicating copy to clipboard operation
pipecat copied to clipboard

openai realtime beta

Open kwindla opened this issue 4 months ago • 1 comments

Okay, I think this is ready for review.

Function calling has been tested pretty heavily. Most things are configurable.

I've wired up RTVI/configuration frames, though I have not tested much of that.

I also added a decent start on saving/loading context portably between LLMs. The idea is to standardize on the OpenAI messages format, converting to/from when we save/load. This is going be rife with corner cases, but I've tested openai, openai-realtime, and anthropic. Basic messages and function calling work robustly.

In examples/foundational

20a-persistent-context-openai.py
20b-persistent-context-openai-realtime.py
20c-persistent-context-anthropic.py

kwindla avatar Oct 02 '24 21:10 kwindla