async-openai icon indicating copy to clipboard operation
async-openai copied to clipboard

Add HTTP Client trait abstraction for middleware support

Open timvw opened this issue 3 months ago • 0 comments

Summary

This PR introduces an HttpClient trait abstraction that allows users to provide custom HTTP client implementations, enabling middleware support for automatic instrumentation, logging, retry logic, and more.

Motivation

Currently, async-openai is tightly coupled to reqwest::Client, which prevents users from:

  • Adding middleware for tracing, logging, or retry logic
  • Using alternative HTTP clients
  • Mocking HTTP calls for testing
  • Implementing custom request/response handling

This is particularly important for production applications that need automatic OpenTelemetry instrumentation without manually wrapping every API call.

Changes

  1. New http_client module (src/http_client.rs):

    • Defines HttpClient trait for abstract HTTP operations
    • HttpError and HttpResponse types for trait methods
    • Default implementation for reqwest::Client
    • BoxedHttpClient type alias for convenience
  2. New ClientWithTrait struct (src/client.rs):

    • Alternative client that accepts any HttpClient implementation
    • Maintains backward compatibility - existing Client unchanged
    • Enables dependency injection of HTTP clients

Benefits

  • Middleware Support: Use reqwest-middleware for automatic OpenTelemetry/tracing
  • Testing: Easy mocking with custom HttpClient implementations
  • Flexibility: Support for any HTTP client library
  • Backward Compatible: Existing code continues to work unchanged

Example Usage

use async_openai::{ClientWithTrait, http_client::HttpClient};
use reqwest_middleware::{ClientBuilder, ClientWithMiddleware};
use reqwest_tracing::TracingMiddleware;

// Create HTTP client with middleware
let client = ClientBuilder::new(reqwest::Client::new())
    .with(TracingMiddleware::default())
    .build();

// Use with async-openai
let openai = ClientWithTrait::new_with_http_client(client, config);

Testing

The changes compile successfully and maintain full backward compatibility. The new trait has been tested with reqwest-middleware for OpenTelemetry instrumentation.

Next Steps

If this abstraction is accepted, future work could:

  • Migrate internal API implementations to use the trait
  • Add built-in middleware implementations
  • Provide testing utilities with mock clients

timvw avatar Aug 22 '25 09:08 timvw