proxy-tuning

proxy-tuning copied to clipboard

Thanks for your great work! I have some questions regarding Proxy-Tuning for GPT-3.5: 1. Since GPT-3.5-turbo-0613 is a chat model rather than a base model (like davinci-002), how can the...

NuoJohnChen

Is there an open-source model that can map its vocabulary to GPT-4o's vocabulary?

NuoJohnChen

I use Qwen-2-0.5b as anti_expert_model, Qwen-2-0.5b tuned on codex_humaneval as expert_model, and Qwen2-7B as base_model, the EM score of proxy-tuned Qwen2-7B is: 0.4167682926829268. ``` # Evaluating DExperts with codex_humaneval expert...

NuoJohnChen

how can i reproduce the results on truthfulqa?

1

I notice that operating truthfulqa.sh requires "gpt_true_model_name" and "gpt_info_model_name". But it seems the original model is unavailable now.

SuperChanS

Hi @alisawuffles, thanks for you novel work! I have a question about the evaluation indicator. When you evaluate models on triviaqa datasets, you calculate the accuracy as follows: test_df['output'] =...

taltalw