litellm icon indicating copy to clipboard operation
litellm copied to clipboard

fix(gemini): append empty choice to model response when there are no …

Open sonald opened this issue 7 months ago • 4 comments

Title

make sure response has at least one choice

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • [x] My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

gemini-2.5-pro-preview-03-25 at some cases will respond with a non-error message that has neither candidates nor promptFeedback, like this:

{
  "usageMetadata": {
    "promptTokenCount": 9291,
    "totalTokenCount": 9291,
    "promptTokensDetails": [
      {
        "modality": "TEXT",
        "tokenCount": 9291
      }
    ]
  },
  "modelVersion": "gemini-2.5-pro-preview-03-25"
}

this breaks apps like smolagents, this is a related issue on smolagents.

sonald avatar Apr 21 '25 10:04 sonald

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 22, 2025 7:40am

vercel[bot] avatar Apr 21 '25 10:04 vercel[bot]

Hey @sonald can you add a unit test for this change in here

krrishdholakia avatar Apr 21 '25 15:04 krrishdholakia

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Apr 22 '25 07:04 CLAassistant

@krrishdholakia I have added the test

sonald avatar Apr 25 '25 11:04 sonald

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

github-actions[bot] avatar Jul 25 '25 00:07 github-actions[bot]