Only has 0.44 accuracy on GSM8K after running the provided codes

[TrueNobility303]

Dear authors,

I train the CLLM model on GSM8k with Abel-7B-001 as the teacher model, using the dataset cleaned_gsm8k_jacobi dataset you provided on huggingface, and run the train_cllm.sh, and set "use_gt_labels" in the file train_cllm_global.py to be False according to this previous issue.

The trained model only has an accuracy 0.44 after running bash eval/gsm8k/acc.sh, which is much lower than the result of the checkpoint provided by you.

Could you tell me what is wrong? What is the exact hyperparameter to reproduce the results?

I would greatly appreciate it if you could help me.

Best regards.