Nick Sciarrilli
Nick Sciarrilli
Did you check your IAM permissions? I am able to access without issue.
+1 to this. The examples for model_config from the github examples folder are not working
@NathanHB it seems like none of these [examples](https://github.com/huggingface/lighteval/blob/main/examples/model_configs/transformers_model.yaml) are following the correct yaml structure. I see a different structure in the [docs](https://huggingface.co/docs/lighteval/en/use-vllm-as-backend): ```python model: # Model specific parameters base_params: model_args:...