How to use Mıstral API

[codelauncher444]

How to use Mıstral API as an llm for mistral medium model? Can I get an example code. I am using it like that: llm = dspy.OpenAI(api_base='https://api.mistral.ai/v1/', api_key='MY_API_KEY', model='mistral-medium', model_type='chat') However, error occurs and 422 Unprocessable Entity error specifically points out that frequency_penalty, n, and presence_penalty are not permitted in the request body. Mistral API does not allow these ones as parameter

[arnavsinghvi11]

Hi @codelauncher444 , for now, you can remove these default args in the GPT3 class like I've done here: https://github.com/stanfordnlp/dspy/pull/430/files#diff-777affe74e42de59a59f0918a7e36139b2186f27bc696b241afef40cfa6af453R70 while testing Mistral.

@okhat It seems that not all available OpenAI API SDK endpoints are compatible with all the default args we have set in GPT3. Should we refactor the default kwargs only to include temperature and max_tokens potentially, and leave the rest as configurable by the user?

[JLKaretis]

I've had some success using a custom class like this, for Mistral models deployed with Azure. This isn't fully tested, results might vary

`from mistralai.client import MistralClient from mistralai.models.chat_completion import ChatMessage import os from dsp import LM from typing import Any

class Mistral(LM):

def __init__(self, model, api_key, endpoint, **kwargs):
    self.model = model
    self.api_key = api_key
    self.endpoint = endpoint 
    self.client = MistralClient(endpoint=self.endpoint, api_key=self.api_key)
    self.kwargs = {
        "temperature": 0.0,
        "max_tokens": 150,
        "top_p": 1,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "n": 1,
        **kwargs,
    }
    self.history: list[dict[str, Any]] = []
    self.provider = "openai"

def basic_request(self, prompt: str, **kwargs):
    chat_response = self.client.chat(
        model=self.model,
        messages=[
            ChatMessage(role="user", content=prompt)
        ],
        **kwargs
    )
    response_dict = json.loads(chat_response.model_dump_json())

    self.history.append({
        "prompt": prompt,
        "response": response_dict, #{"choices": chat_response.choices[0].message.content},
        "kwargs": kwargs,
    })

    return chat_response

def __call__(self, prompt, only_completed=True, return_sorted=False, **kwargs):
    response = self.request(prompt, **kwargs)
    completions = [response.choices[0].message.content] 
    return completions

def _get_choice_text(self, choice: dict[str, Any]) -> str:
    
    return choice["message"]["content"]`