code-eval

code-eval copied to clipboard

Any plans on running evals for codellama?

2

I'm keeping https://github.com/ErikBjare/are-copilots-local-yet up-to-date, and would love to see some codellama numbers given it's now SOTA :)

ErikBjare


zhimin-z

Is llama2-7B-chat weaker thank llama2-7B?

1

I got only 9.7% for llama2-7B-chat on human-eval using your script ``` python {'pass@1': 0.0975609756097561} ```

sunyuhan19981208

Support CodeGeeX2

2

建议支持 https://github.com/THUDM/CodeGeeX2 刚刚发布，根据公布1@pass数据达到了35.9

exceedzhang

invade-art