evals
evals copied to clipboard
Multiple Unit Test Failures Across OpenAI Assistants, Anthropic, and Google Gemini Libraries
Describe the bug
I observed several failures in unit tests involving the libraries for OpenAI Assistants, Anthropic, and Google Gemini. Each, detailed below.
-
Anthropic Library:
-
Issue: Recent changes have made the
ContentBlocktype a Union type, which can't be directly instantiated. This updated type now encapsulates bothTextBlockandToolUseBlock.ContentBlock = Annotated[Union[TextBlock, ToolUseBlock], PropertyInfo(discriminator="type")] -
Impact: Our existing code relies on the direct instantiation of
ContentBlock, which is now causing failures.
-
Issue: Recent changes have made the
-
OpenAI Library:
-
Issue: The latest update to the assistant API introduced several breaking changes:
- The
retrievaltool has been renamed tofile_search. - The parameter
assistant.file_idshas been changed totool_resources. - The parameter
message.file_idshas been modified toattachments.
- The
- Impact: These changes are causing failures in the functionalities that depend on file handling and assistant resources.
-
Issue: The latest update to the assistant API introduced several breaking changes:
-
Gemini Library:
- Issue: There's a defect in the Gemini library related to how it handles protobuf objects; specifically, it erroneously parses these objects as dictionaries.
- Impact: This parsing error is causing unexpected behavior and test failures.
To Reproduce
Just run the unit tests for v 3.0.1
Code snippets
No response
OS
macOS
Python version
3.9.0
Library version
3.0.1