[DRAFT] FEAT migrate to openai SDK for openai API targets
Description
Migrating (back) to the openai SDK since it appears to have matured to a point where we can actually use it. This simplifies some things for us as we don't have to manually craft HTTP messages anymore.
The PR handles this change and adds more structure for error handling. This can be observed in the OpenAITarget abstract base class with _handle_openai_request which is called by each target. In turn, it calls the provided API (this is specific per target, e.g., responses API, or chat completions API), catches any exceptions (e.g., input content filters, bad requests, rate limit error, and others), and if it succeeds runs _check_content_filter (for output filters) followed by _handle_content_filter_response in case of filters. It also validates the response with _validate_response and finally returns the result (which may be an error Message or a successful Message).
Note: OpenAIRealTimeTarget is an exception as it doesn't use the openai SDK. There's no support for that.
Note: this is pending another thorough re-review by myself and then a review from other maintainers.
Note: This does not yet add token provider capabilities instead of the API key. That will require a follow-up PR.
Tests and Documentation
- Unit tests ✅
- pre-commit ✅
- integration tests (also added new ones)
- general for targets ✅
- content filters ✅
- ollama (tbd)
- notebooks (tbd)