openai-api-rs
openai-api-rs copied to clipboard
dongri
Async support
Describe the feature or improvement you're requesting
Most real-time applications are likely to use Tokio or any other async runtime. The current Client does not support async.
Additional context
No response
The reason for excluding dependencies in Tokio https://github.com/dongri/openai-api-rs/pull/31
I see, but would it hinder those who require an async version (most production-ready apps nowadays rely on Tokio or similar runtimes for async futures)?
I understand the concern of the size of the library and build times, but AFAIK, these could be reduced if the feature set of Tokyo is appropriately selected based on what's used. I've noticed that the previous version used to import full set of features from tokio. Perhaps including only the essential features would help to reduce size dramatically without breaking compatibility with async Rust?
That being said, if it's a design decision only to support blocking mode and synchronous API, i.e. the goal of a library (for simple small apps with as simple set up as possible), then I can understand that for sure!
Thank you for your understanding! I think it's fine to use synchronous processing (openai-api-rs) within asynchronous processing (user's application). For example, in the following case: https://github.com/dongri/openai-cli/blob/master/src/main.rs Or is it intended for a different use?
For sure! This is certainly possible, although the provided example code is not suitable for real-world use as such usage would block the runtime/task and so it is discouraged by Tokio (it blocks the thread waiting on I/O without letting the runtime yield to let other tasks run), but there are ways to run blocking code in Tokio, it's just a bit more cumbersome 🙂
I see, it's about calling the OpenAI API and allowing other processes to run without blocking while waiting, in other words, asynchronously. Let me think about it. Should I fork and create openai-api-async-rs lol?
Using a library like https://github.com/64bit/async-openai might be a good idea
Using a library like https://github.com/64bit/async-openai might be a good idea
That's right, that's what I ended up using in a project that runs in production 🙂
I still use openai-api-rs for quick small when doing experiments, though.
That being said, it's possible to have both a blocking and an async version of a crate. Typically, the common part is designed in a sans-io way, and then there are features like tokio and sync that enable modules that add sync or tokio high-level wrappers correspondingly. That way, users have a choice between blocking and async versions depending on their needs.
@daniel-abramov Thank you very much for your patience. After careful consideration, we have decided to support async. We also changed to using reqwest due to the issue of minreq being unable to send multipart. Please use version v5.0.0