how can i reproduce the results on truthfulqa?

[SuperChanS]

I notice that operating truthfulqa.sh requires "gpt_true_model_name" and "gpt_info_model_name". But it seems the original model is unavailable now.

[alisawuffles]

Yes, this part of the evaluation is unfortunately no longer reproducable due to OpenAI's deprecation of GPT-3-based models. The allenai/open-instruct evaluation frameowrk has switched to finetuned judge models based on Llama2 instead! Please see their evaluation script here.