bigcodebench icon indicating copy to clipboard operation
bigcodebench copied to clipboard

Is it normal for the ground truth accuracy not to be 100%?

Open p81sunshine opened this issue 7 months ago • 0 comments

Image

Under this setting, my evaluation results on qwen2.5coder-instruct-3b is betther than results claimed from the officical techinique report.

Image

p81sunshine avatar Jul 14 '25 04:07 p81sunshine