[Feature]: FIM endpoint for Mistral

[azdolinski]

The Feature

there is now: "text-completion-codestral" prefix to use fim/completions but there is no way to use FIM endpoint: https://api.mistral.ai/v1/fim/completions

proposal: add "text-completion-mistral" which will point to https://api.mistral.ai/v1/fim/completions

Mistra FIM docs: Link

Motivation, pitch

use new models from mistral for fim tab autocomplete

Mistral models: Link

Twitter / LinkedIn details

https://www.linkedin.com/in/artur-zdolinski/

[nomarek]

Are there any plans for implementing this?

[azdolinski]

@nomarek my workaround:

  - model_name: text-completion-mistral-open-codestral-mamba
    litellm_params:
        model: text-completion-codestral/open-codestral-mamba
        api_base: https://api.mistral.ai/v1/fim/completions
        api_key: os.environ/MISTRAL_API_KEY

  - model_name: text-completion-mistral-codestral-latest
    litellm_params:
        model: text-completion-codestral/codestral-latest
        api_base: https://api.mistral.ai/v1/fim/completions
        api_key: os.environ/MISTRAL_API_KEY

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[mvrodrig]

@nomarek my workaround:

• model_name: text-completion-mistral-open-codestral-mamba litellm_params: model: text-completion-codestral/open-codestral-mamba api_base: https://api.mistral.ai/v1/fim/completions api_key: os.environ/MISTRAL_API_KEY

• model_name: text-completion-mistral-codestral-latest litellm_params: model: text-completion-codestral/codestral-latest api_base: https://api.mistral.ai/v1/fim/completions api_key: os.environ/MISTRAL_API_KEY

Hi @azdolinski , Could you share a sample cURL to test this using the proxy? Thanks!