Request - Integrate with Litellm

[ranjancse26]

I believe, it's a great idea to incorporate the Litellm as it already supports Intergrations with 100+ LLMs.

https://github.com/BerriAI/litellm

LLM as drop-in replacement for gpt-3.5-turbo. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs)

cc @ishaan-jaff

[jeremi]

This is what is used in the latest version of the code at least.