finetune accuracy is much higher than what is in the README table

[CrazyElements]

Great work, I use the fine-tune code that can be obtained in math_running_commands to train the pre-trained model yahma/llama-7b-hf with LoRA. And then I evaluate the fine-tuned model on SVAMP, whose accuracy is 66.2%, with about 40% improvement form the previous 47.2%.

Could you please tell me why I can get a much better result? Did you update the dataset again?

Thank you!

[HZQ950419]

Hi, yes, we have updated the dataset. And We will update the table soon. Sorry for the confusion!

[ToheartZhang]

@HZQ950419 Hi, I wonder if the updated results are obtained by models trained on math_10k.json or another training dataset?

[HZQ950419]

@HZQ950419 Hi, I wonder if the updated results are obtained by models trained on math_10k.json or another training dataset?

Hi, the updated results are obtained by models trained on math_10k.json for math reasoning.

[HZQ950419]

Great work, I use the fine-tune code that can be obtained in math_running_commands to train the pre-trained model yahma/llama-7b-hf with LoRA. And then I evaluate the fine-tuned model on SVAMP, whose accuracy is 66.2%, with about 40% improvement form the previous 47.2%.

Could you please tell me why I can get a much better result? Did you update the dataset again?

Thank you!

Hi,

After carefully checking, I can't reproduce your results with llama-7b-lora. Could you give more information? The command used for training and testing, the hardware used for the experiment, and so on.

Thanks!