Facing issue while running different Hugging Face Models

[Hizafa-Nadeem]

I am unable to run tests for different Hugging Face models like databricks/dolly-v2-3b, openlm-research/open_llama_3b, AlekseyKorshuk/vicuna-7b and more. I have used 'BoolQ-test-tiny' dataset for robustness test.

harness.run()

harness.run() function keeps on executing and doesn't give any output.

[chakravarthik27]

I think you need to have access to GPU configuration from hugging face to handle it. and it's not a problem in nlptest but it is in the hugging face platform also.

if the model was loaded successfully in the below link, then it will work with nlptest also. https://huggingface.co/openlm-research/open_llama_3b?text=My+name+is+Thomas+and+my+main