OpenAI-DotNet
OpenAI-DotNet copied to clipboard
RageAgainstThePixel
Add Realtime WebRTC Support
@StephenHodgson did you have any particular approach in mind for WebRTC? Did you want to include everything required into this package?
The reason I ask is I'm coming fom the opposite direction and am looking into how best to include an OpenAI library into some WebRTC demos. The demos are all working but I'm keen to avoid maintaining yet another set of OpenAI models, serialisation etc.
@sipsorcery good questions. I have been using your sip packages for my own projects but have not done any WebRTC stuff.
If you have any guidance here please let me know, because I already use your packages as deps in my other OpenAI projects.
If you have any guidance here please let me know, because I already use your packages as deps in my other OpenAI projects.
Off the top of my head in decreasing order of preference:
- Add WebRTC Realtime support direcly into one of the existing OpenAI packages such as this one.
- Create a new WebRTC Realtime package "next" to one of the existing OpenAI packages such as this one.
- Create a new WebRTC Realtime package in the SIPSorcery project.
At the moment the OpenAI Realtime endpoint only supports audio which means no native dependencies are required. That would make it straight forward to add to an existing package although the dependency count would go up.
One caveat is that the SIPSorcery package does currently have 7 classes that are at least partial licensed by GPL derivatives. Until that encumberance is removed by an alternative implementation it could be better to go with the second option and have a new package.
Just thinking out loud really and I'd be completely open to any suggestions tbh.
Unfortunately I don't have bandwidth to commit to this in the near future, but I could see myself getting around to it on weekends when I've got some free time.
I've been putting off the WebRTC implementation for some time, and at least stubbing in the classes and interconnectivity should give you a head start on your side if you're just wanting to take the dependency on this library.
What was the interest with using this library vs the msft/openai official dotnet version? (they did give me and a few other dotnet devs shoutouts for our community work while they didn't have an official offering)
I mainly keep this one around for my Unity projects since there is no library available there, and I need a backend proxy redirector.
Out of all the preferences, the first seems like the most appropriate.
Oh, I'm happy to do, or at least submit a PR for the WebRTC implementation. The main work is done it just needs to be massaged into whichever library it sits in.
What was the interest with using this library vs the msft/openai official dotnet version?
That seems to be the least active and clunkiest library tbh, seems like the maintainer is already fading. I did start on an attempt for a PR in the betalgo library but I'm struggling a bit to resonate with the code structure (most likely my deficiency rather than the library). This library seems simpler and cleaner to me and would be easier for me to work with.
I'll switch to making a PR for this library and see how I go. It's mostly weekend resources for me as well.
Yes, I try to be as active and responsive to all my projects as possible, so if you open a PR I am more than willing to work with you to get it integrated. Also mad respect for your existing work since I have been using some of your libraries for some time. 🍻
I'll go ahead and add you as a collaborator so you can make a branch and submit pull requests so that unit tests work correctly.
please use Draft option when opening a PR so that unit tests only run when we're ready to do final checks and merges since it costs money to run unit tests against API calls.