Added support to additional MIME types for Gemini model

[NiharP31]

Title

Add support for additional MIME types in Gemini models

Relevant issues

Fixes #9416

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the [tests/litellm/](https://github.com/BerriAI/litellm/tree/main/tests/litellm) directory, Adding at least 1 test is a hard requirement - [see details](https://docs.litellm.ai/docs/extras/contributing_code)
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on (make test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature

Changes

This PR adds support for non-image/PDF file types with Gemini models, specifically addressing:

1. Fixed handling of text-based formats (Markdown, JSON, XML, CSV) that don't require base64 encoding
2. Resolved token counting errors for text files by preventing unnecessary encoding
3. Expanded MIME type detection to support additional file formats
4. Maintains backward compatibility with existing image/PDF handling

Files Modified:

• litellm/types/files.py:

  • Added requires_base64_encoding() function
  • Added new file types (Markdown, XML) to supported types
  • Updated GEMINI_1_5_ACCEPTED_FILE_TYPES to include text-based formats

• litellm/llms/vertex_ai/gemini/transformation.py:

  • Renamed _process_gemini_image() to _process_gemini_file()
  • Added conditional logic to handle text files without base64 encoding
  • Expanded _get_mime_type_from_url() to detect additional formats
  • Fixed typo in error message ("filr" → "file")
  • Updated function references

Files Created:

• tests/litellm/llms/vertex_ai/gemini/test_transformation.py:
  • Added tests for MIME type detection
  • Added tests for text vs binary file handling
  • Added tests for file type validation

Images:

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 27, 2025 9:00pm