gemma_pytorch icon indicating copy to clipboard operation
gemma_pytorch copied to clipboard

Published 20 hours ago • verified publisher icon google

Unable to reproduce MATH resulst

Open wenhuchen opened this issue 3 months ago • 2 comments

Hi there, thanks for sharing gemma. But it seems that I can't reproduce the MATH 24% 4-shot accuracy. I'm only getting 20% now. Is there anyone trying to reproduce that? What's the prompt?

wenhuchen avatar Mar 23 '24 03:03 wenhuchen