[Bug]: Proxying OpenAI client to Ollama results in internal server error due to badly formed options json

[merefield]

What happened?

Running LiteLLM as a local proxy from an OpenAI compatible client targetting Ollama Gemma which causes a very strange issue with options, specifically, the json seems badly formed:

'options': {'temperature': 0.2, 'top_p': 1.0, 'num_predict': 200, 'repeat_penalty': 'frequency_penalty'},

so the call fails with internal server error 500.

This affects 1.34.39 but seems was also an issue on 1.34.0

Relevant log output

LiteLLM: Proxy initialized with Config, Set models:
    gpt-3.5-turbo
07:39:46 - LiteLLM Router:INFO: router.py:289 - Intialized router with Routing strategy: simple-shuffle

Routing fallbacks: None

Routing context window fallbacks: None
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)
07:40:12 - LiteLLM:INFO: utils.py:1112 - 

POST Request Sent from LiteLLM:
curl -X POST \
http://localhost:11434/api/chat \
-d '{'model': 'gemma', 'messages': [{'role': 'system', 'content': 'You are a helpful assistant. When referring to users by name, include an @ symbol directly in front of their username.'}, {'role': 'assistant', 'content': 'The subject of this conversation is A Discussion with Chatbot'}, {'role': 'assistant', 'content': 'The first thing someone said was Chatbot who said Hello, how can I help you?'}, {'role': 'assistant', 'content': 'Hello, how can I help you?'}, {'role': 'user', 'content': 'merefield said what colour is a london bus'}], 'options': {'temperature': 0.2, 'top_p': 1.0, 'num_predict': 200, 'repeat_penalty': 'frequency_penalty'}, 'stream': False}'


Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/litellm/llms/ollama_chat.py", line 366, in ollama_acompletion
    raise OllamaError(status_code=resp.status, message=text)
litellm.llms.ollama_chat.OllamaError: {"error":"option \"repeat_penalty\" must be of type float32"}

SNIP

litellm.exceptions.APIConnectionError: {"error":"option \"repeat_penalty\" must be of type float32"}
INFO:     127.0.0.1:62230 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error

Twitter / LinkedIn details

No response