litellm icon indicating copy to clipboard operation
litellm copied to clipboard

Added support to additional MIME types for Gemini model

Open NiharP31 opened this issue 9 months ago • 2 comments

Title

Add support for additional MIME types in Gemini models

Relevant issues

Fixes #9416

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • [x] I have Added testing in the [tests/litellm/](https://github.com/BerriAI/litellm/tree/main/tests/litellm) directory, Adding at least 1 test is a hard requirement - [see details](https://docs.litellm.ai/docs/extras/contributing_code)
  • [x] I have added a screenshot of my new test passing locally
  • [ ] My PR passes all unit tests on (make test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]
  • [x] My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature

Changes

This PR adds support for non-image/PDF file types with Gemini models, specifically addressing:

  1. Fixed handling of text-based formats (Markdown, JSON, XML, CSV) that don't require base64 encoding
  2. Resolved token counting errors for text files by preventing unnecessary encoding
  3. Expanded MIME type detection to support additional file formats
  4. Maintains backward compatibility with existing image/PDF handling

Files Modified:

  • litellm/types/files.py:

    • Added requires_base64_encoding() function
    • Added new file types (Markdown, XML) to supported types
    • Updated GEMINI_1_5_ACCEPTED_FILE_TYPES to include text-based formats
  • litellm/llms/vertex_ai/gemini/transformation.py:

    • Renamed _process_gemini_image() to _process_gemini_file()
    • Added conditional logic to handle text files without base64 encoding
    • Expanded _get_mime_type_from_url() to detect additional formats
    • Fixed typo in error message ("filr" → "file")
    • Updated function references

Files Created:

  • tests/litellm/llms/vertex_ai/gemini/test_transformation.py:
    • Added tests for MIME type detection
    • Added tests for text vs binary file handling
    • Added tests for file type validation

Images:

litellm1

NiharP31 avatar Mar 27 '25 20:03 NiharP31

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 27, 2025 9:00pm

vercel[bot] avatar Mar 27 '25 20:03 vercel[bot]

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Apr 22 '25 22:04 CLAassistant

@NiharP31 I see that this PR is stale for a few months. Do you plan to fix the conflicts so that it can be merged? Does it support Gemini 2.5 models?

shahar3012 avatar Jul 10 '25 09:07 shahar3012

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

github-actions[bot] avatar Oct 09 '25 00:10 github-actions[bot]