evals icon indicating copy to clipboard operation
evals copied to clipboard

Fix Unit Test Failures in OpenAI, Anthropic, and Google Gemini Resolvers

Open sakher opened this issue 1 year ago • 1 comments

Fixed all errors in #1536

This pull request addresses multiple failures in the unit tests for the libraries used in OpenAI Assistants, Anthropic, and Google. Here are the key changes made to resolve these issues:

Bug Description:

  1. Anthropic Library:

    • Issue: The type ContentBlock has been updated to a Union type which cannot be instantiated. It now represents both TextBlock and ToolUseBlock.
      ContentBlock = Annotated[Union[TextBlock, ToolUseBlock], PropertyInfo(discriminator="type")]
      
    • Fix: Replaced ContentBlock with TextBlock in the relevant sections of the code.
  2. OpenAI Library:

    • Issue: The assistant API has breaking changes:
      • retrieval tool renamed to file_search
      • assistant.file_ids parameter changed to tool_resources
      • message.file_ids parameter changed to attachments
    • Fix: Updated the code to reflect these changes.
  3. Gemini Library:

    • Issue: An error exists in the Gemini library where it incorrectly parses a protobuf object as a dictionary.

    • Fix: Implemented a workaround to avoid this parsing bug.

    • Reference:

      • https://github.com/google-gemini/generative-ai-python/blob/419a7cef81188a2d715b3853113a3213516d6a4a/google/generativeai/types/generation_types.py#L416
      • I believe it is this issue: https://github.com/google-gemini/generative-ai-python/issues/288

To Reproduce: Run the unit tests for version 3.0.1 to observe the failures.

Changes Made:

  • Replaced ContentBlock with TextBlock in the Anthropic library code.
  • Updated the assistant API in the OpenAI library to use the new parameters and tool names.
  • Added a workaround to handle the protobuf parsing issue in the Gemini library.

Environment:

  • OS: macOS
  • Python Version: 3.9.0
  • Library Version: 3.0.1

This pull request ensures that all unit tests pass successfully, and the libraries are compatible with the latest updates and changes.

image

sakher avatar Jun 24 '24 18:06 sakher

F

Hi @ikrnwath sorry but I am not sure what you mean. Please let me know if I need to make any changes .

sakher avatar Jul 07 '24 20:07 sakher