Adam Hendel

I'd love to see this go out!

I wanted to drop it here so you knew I was starting to work on it. If there are glaring issues with it, please let me know :) I'll move...

I'm able to get the jupyterhub container running, but looks like I broke some things in the choochoo build along the way :(

Hi @drazvan - is there a way to set the `openai_api_base`, `model_name`, and authorization header values via environment variables? Specifically when `engine: vllm_openai`

Thanks! That gave me another idea which ended up working too, it seems. Any concerns doing it this way? ```python import os from nemoguardrails.rails.llm.config import Model from nemoguardrails import LLMRails,...

I should add that `/path/to/config.yaml`, in my example above, does not contain any values for `models`, but it does seem to work the way I want it to.

> What is CHAT_MODEL here ? The name of HF model ? Yes `CHAT_MODEL` is the name of the model, e.g. `meta-llama/Meta-Llama-3-8B-Instruct` > Is vLLM model created under hood ?...

@snassimr , vLLM is an alternative to OpenAI. You could run vLLM on your local machine or on a cloud VM, for example. vLLM doesn't require the API key, but...