TabLLM icon indicating copy to clipboard operation
TabLLM copied to clipboard

The scores after every 30 epochs doesn't change at all during fine-tuning

Open tanayshah23 opened this issue 1 year ago • 1 comments

Hi,

I am trying to fine-tune the t03b model with a 4-shot approach on the heart dataset. On checking the logs after every 30 epochs, I don't see any difference in the scores that are printed. It prints the same score after the first 30 epochs and the same at the end of 5600 epochs

{"AUC": 0.5823586744639375, "PR": 0.6555849741948994, "micro_f1": 0.5706521739130435, "macro_f1": 0.4795001253267447, "accuracy": 0.5706521739130435, "num": 184, "num_steps": -1, "score_gt": 0.3382242697736491, "score_cand": 0.37344987593267276}
....
....
....
{"AUC": 0.5823586744639375, "PR": 0.6555849741948994, "micro_f1": 0.5706521739130435, "macro_f1": 0.4795001253267447, "accuracy": 0.5706521739130435, "num": 184, "num_steps": 30, "score_gt": 0.3382242697736491, "score_cand": 0.37344987593267276}

I also tried the same thing on a different dataset and got the same results. Here's the screenshot: image

Can you please tell me what am I doing wrong?

tanayshah23 avatar Oct 29 '23 18:10 tanayshah23