async-openai
async-openai copied to clipboard
Add HTTP Client trait abstraction for middleware support
Summary
This PR introduces an HttpClient trait abstraction that allows users to provide custom HTTP client implementations, enabling middleware support for automatic instrumentation, logging, retry logic, and more.
Motivation
Currently, async-openai is tightly coupled to reqwest::Client, which prevents users from:
- Adding middleware for tracing, logging, or retry logic
- Using alternative HTTP clients
- Mocking HTTP calls for testing
- Implementing custom request/response handling
This is particularly important for production applications that need automatic OpenTelemetry instrumentation without manually wrapping every API call.
Changes
-
New
http_clientmodule (src/http_client.rs):- Defines
HttpClienttrait for abstract HTTP operations HttpErrorandHttpResponsetypes for trait methods- Default implementation for
reqwest::Client BoxedHttpClienttype alias for convenience
- Defines
-
New
ClientWithTraitstruct (src/client.rs):- Alternative client that accepts any
HttpClientimplementation - Maintains backward compatibility - existing
Clientunchanged - Enables dependency injection of HTTP clients
- Alternative client that accepts any
Benefits
- Middleware Support: Use
reqwest-middlewarefor automatic OpenTelemetry/tracing - Testing: Easy mocking with custom
HttpClientimplementations - Flexibility: Support for any HTTP client library
- Backward Compatible: Existing code continues to work unchanged
Example Usage
use async_openai::{ClientWithTrait, http_client::HttpClient};
use reqwest_middleware::{ClientBuilder, ClientWithMiddleware};
use reqwest_tracing::TracingMiddleware;
// Create HTTP client with middleware
let client = ClientBuilder::new(reqwest::Client::new())
.with(TracingMiddleware::default())
.build();
// Use with async-openai
let openai = ClientWithTrait::new_with_http_client(client, config);
Testing
The changes compile successfully and maintain full backward compatibility. The new trait has been tested with reqwest-middleware for OpenTelemetry instrumentation.
Next Steps
If this abstraction is accepted, future work could:
- Migrate internal API implementations to use the trait
- Add built-in middleware implementations
- Provide testing utilities with mock clients