Zixuan Cheng comments

Results 3 comments of


                                            Zixuan Cheng

Request for CodeLlama's Specific Production Parameters on Human-Eval Dataset

Hi @itaigat , I've been trying to reproduce CodeLlama-7b pass scores on HumanEval in the paper. I used [bigcode-evaluation-harness](https://github.com/bigcode-project/bigcode-evaluation-harness) for evalutaion with HumanEval task, using evaluate's [code_eval](https://github.com/huggingface/evaluate/blob/main/metrics/code_eval/code_eval.py) as metric. The...

Model execution is single threaded?

Hi @akhauriyash , I was not able to reproduce this issue on several machines yet. Could you please share your enviroment where the issue occurs using `pip list`?

Export Quantized Model to ONNX: `NotImplementedError: Could not run 'quantized::conv2d.new' with arguments from the 'CPU' backend.`

Hi @clementpoiret , as @yuwenzho mentioned above, quantized `transpose` is not supported by ONNX exporter. For the full supported/unsupported TorchScript operators by ONNX export, please refer to [ONNX SUPPORTED TORCHSCRIPT...