MoE-PEFT icon indicating copy to clipboard operation
MoE-PEFT copied to clipboard

Published 20 hours ago • verified publisher icon TUDB-Labs

A question about the evaluation code

Open air-tea opened this issue 4 months ago • 0 comments

Thanks for your excellent work!

I have a question about the evaluation code. The evaluation code extracts the last token in the generated answer of the LLM and uses it to compare it with the ground truth label. Is this setting a comment setting in the evaluation of tasks such as single-label classification? What if the generated answer has the label in the middle of it?

air-tea avatar Oct 09 '24 15:10 air-tea