models icon indicating copy to clipboard operation
models copied to clipboard

Can't get comparable accuracy

Open mengniwang95 opened this issue 2 years ago • 0 comments

Bug Report

Which model does this pertain to?

https://github.com/onnx/models/tree/main/vision/object_detection_segmentation/retinanet https://github.com/onnx/models/tree/main/text/machine_comprehension/roberta

Describe the bug

For retinanet, with preprocess and postprocess code in readme, I can't get mAP 0.376. For roberta, I use RoBERTa-SequenceClassification model and MRPC dataset and calculate GLUE, its drop a lot (it is 90.2 in readme)

Reproduction instructions

System Information

OS Platform and Distribution (e.g. Linux Ubuntu 16.04):
ONNX version (e.g. 1.6): 1.10.0 Backend/Runtime version (e.g. ONNX Runtime 1.1, PyTorch 1.2): ORT 1.11.0

Notes

Any additional information

mengniwang95 avatar Sep 28 '22 05:09 mengniwang95