Expose more pub(crate) methods as public or provide traits to override behavior

[ryanolson]

First off, this is a really well done project. Thank you for all your work.

There is one feature missing that is preventing me from fully scratching my current OAI bits and moving to this project.

I have an OAI service, but I have added new behavior to the SSE event stream to augment the information sent back in the streaming response. Besides just using the data: field in the SSE stream, I use SSE events and comments to provide a more rich description of the server-side state.

You already have the abstractions needed: https://github.com/64bit/async-openai/blob/main/async-openai/src/client.rs#L493

With this method, you can provide an event handler, which could handle event:, etc.

In my specific case, my output stream would be an Annotated<O> where all the sole data: message are O and all the events and other bits are wrapped by the Annotated wrapper. Your abstractions already allow get/post to be generic with respect to O.

Would you be willing to extend the client functionality to provide more custom SSE handling?

What would this look like? I'm not entirely sure. Initially, we could make this method fully public: https://github.com/64bit/async-openai/blob/main/async-openai/src/client.rs#L402

Later, we could decide if/how to use the trait system.

Thoughts?

[ryanolson]

If I had a trait like

#[async_trait]
pub trait AnnotatedStream<I, O> {
    async fn create_annotated_stream(&self, mut request: I) -> Result<AnnotatedStream<O>, OpenAIError>;
}

I could then apply that to the Chat object.

One further thing would be needed. I would need to access the underlying client. We might expose that as via a ClientProvider trait.

[gilljon]

+1 on this

[64bit]

Thanks for your kind words @ryanolson.

I believe your use can can be served by Bring your own types feature?

For example, Threads::create_and_run_thread_byot method gives you access to manipulate SSE events as you need: https://github.com/64bit/async-openai/blob/async-openai-v0.28.0/async-openai/src/threads.rs#L42-L70

If you're looking for something like that in Chat::create_stream_byot, it may require to use the self.client.post_stream_mapped_raw_events instead of self.client.post_stream and associated changes.