MiniCPM icon indicating copy to clipboard operation
MiniCPM copied to clipboard

General Question: Which framework do you use to evaluate MiniCPM3-4B?

Open xunkai55 opened this issue 1 year ago • 0 comments

Hi there,

I tried to evaluate MiniCPM3-4B by my self (deployed locally with OpenBMB/vllm) with OpenAI/simple-evals but got weirdly low scores, i.e. MMLU = 59.8. Before digging it deeper, I'd like to know which evaluation framework do you use.

Thanks a lot!

xunkai55 avatar Sep 13 '24 17:09 xunkai55