Create generic LLM client using LiteLLM

[rbren]

This PR creates a generic client for prompting LLMs. It passes the LLM instance into the Agent constructor, and the Agent can use it however it wants.

This makes it easier for the user to control exactly which foundation models are used.

[shuther]

Just to add my 2 cents, we could still use the openai SDK, and allow the user to modify the base_url. If users wants to use litellm as a proxy/router, they can connect to their litellm (i.e. loaded as a docker) that will connect to ollama or openrouter, vs using openAI endpoint. So just allowing the user to modify the base_url would give us more time to think how to abstract further the LLM. For me, the model to pick depends on the step and the size of the context (if we think about self hosted LLM, the PM could run orca2, the implementation could be done using codellama, the documentation with mistral,...). Litellm has some routing strategy we could leverage later on to do that if we want to.

[rbren]

@xingyaoww after a rough merge, this is ready for re-review 😅