litellm icon indicating copy to clipboard operation
litellm copied to clipboard

Update mistral-medium prices and context sizes

Open regismesquita opened this issue 6 months ago โ€ข 3 comments
trafficstars

While testing the Mistral model, I noticed a discrepancy in the pricing shown on the logs screen. After reviewing the code, I confirmed that the pricing values were incorrect.

This PR corrects the input and output token pricing for the latest Mistral model and adds the newly released mistral-medium-2505 version.

Title

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • [ ] I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • [ ] I have added a screenshot of my new test passing locally
  • [ ] My PR passes all unit tests on make test-unit
  • [ ] My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

๐Ÿ†• New Feature ๐Ÿ› Bug Fix ๐Ÿงน Refactoring ๐Ÿ“– Documentation ๐Ÿš„ Infrastructure โœ… Test

Changes

regismesquita avatar May 10 '25 21:05 regismesquita

The latest updates on your projects. Learn more about Vercel for Git โ†—๏ธŽ

Name Status Preview Comments Updated (UTC)
litellm โœ… Ready (Inspect) Visit Preview ๐Ÿ’ฌ Add feedback May 27, 2025 9:56pm

vercel[bot] avatar May 10 '25 21:05 vercel[bot]

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar May 10 '25 21:05 CLAassistant

@emerzon Mistral medium latest is only 32k?! Every place that I checked shows 128k.

regismesquita avatar May 14 '25 00:05 regismesquita

Does mistral/mistral-medium points to the old one?

regismesquita avatar May 14 '25 07:05 regismesquita

@regismesquita It's probably better to add a new entry for the new model matching the API naming from mistral: mistral-medium-2505 and mistral-medium-latest to avoid removing information from the past model. https://docs.mistral.ai/getting-started/models/models_overview/

Also, mind that the the output context size (max_tokens and max_output_tokens) is not 128k - No clear source but probably still 8k.

emerzon avatar May 14 '25 12:05 emerzon

This very link that you shared shows max tokens at 128k. I know they might be actually talking about the input context there... but going by the word... I have also found sources for output at 128k but none reliable , only stuff like openrouter listing of the new model.

I will remove mistral-medium name from the scope of this PR and retain only the 2503 and the latest identifiers.

I will try to experiment with this model today and also try to contact mistral.

regismesquita avatar May 14 '25 12:05 regismesquita

@emerzon I have reverted mistral/mistral-medium to the old values, and now only the -latest alias and the -2503 aliases are updated/created, I am keeping the 128k value for now while I figure this out, I have asked around and I am trying to confirm the actual values, if I can't find any I will do exploratory requests later today.

regismesquita avatar May 14 '25 12:05 regismesquita

128k is for input tokens - not output tokens.

emerzon avatar May 14 '25 14:05 emerzon

@emerzon can you share how are you sure of that? I couldn't confirm that anywhere in their docs.

regismesquita avatar May 14 '25 14:05 regismesquita

@emerzon haven't got any response yet, and I do think that it is less than 128k but since I haven't got any response from them I prefer to err on the safe side. so I changed it down to 8191 , and I can follow up later if I found any evidence against it.

regismesquita avatar May 14 '25 14:05 regismesquita

Ignore this comment.

regismesquita avatar May 17 '25 09:05 regismesquita

synced with the upstream.

regismesquita avatar May 27 '25 21:05 regismesquita