ROCmSoftwarePlatform
Add ONNX model testing workflow
Summary
This PR creates and updates the onnx-model.yaml workflow to automate model zoo and onnx model testing. Changes
Removed explicit gh_token input (no longer required to run this workflow).
Cleaned up permissions section to avoid unnecessary write scope.
Updated results publishing step:
Commented out direct publish to danieyan-amd/migraphx-reports.
Added a placeholder path (danieyan-amd/migraphx-reports for now) to clarify current output destination.
| Test | Batch | Rate new 38fdc6 |
Rate old 38fdc6 |
Diff | Compare |
|---|---|---|---|---|---|
| torchvision-resnet50 | 64 | 3,158.73 | 3,157.16 | 0.05% | :white_check_mark: |
| torchvision-resnet50_fp16 | 64 | 6,592.62 | 6,584.70 | 0.12% | :white_check_mark: |
| torchvision-densenet121 | 32 | 2,437.74 | 2,436.81 | 0.04% | :white_check_mark: |
| torchvision-densenet121_fp16 | 32 | 4,115.73 | 4,114.72 | 0.02% | :white_check_mark: |
| torchvision-inceptionv3 | 32 | 1,666.40 | 1,665.37 | 0.06% | :white_check_mark: |
| torchvision-inceptionv3_fp16 | 32 | 2,588.05 | 2,587.64 | 0.02% | :white_check_mark: |
| cadene-inceptionv4 | 16 | 794.32 | 794.97 | -0.08% | :white_check_mark: |
| cadene-resnext64x4 | 16 | 802.69 | 802.04 | 0.08% | :white_check_mark: |
| slim-mobilenet | 64 | 8,209.14 | 8,202.93 | 0.08% | :white_check_mark: |
| slim-nasnetalarge | 64 | 221.72 | 221.82 | -0.04% | :white_check_mark: |
| slim-resnet50v2 | 64 | 3,295.61 | 3,285.74 | 0.30% | :white_check_mark: |
| bert-mrpc-onnx | 8 | 1,135.13 | 1,134.61 | 0.05% | :white_check_mark: |
| bert-mrpc-tf | 1 | 487.52 | 487.66 | -0.03% | :white_check_mark: |
| pytorch-examples-wlang-gru | 1 | 315.43 | 312.89 | 0.81% | :white_check_mark: |
| pytorch-examples-wlang-lstm | 1 | 440.33 | 437.57 | 0.63% | :white_check_mark: |
| torchvision-resnet50_1 | 1 | 804.97 | 804.91 | 0.01% | :white_check_mark: |
| cadene-dpn92_1 | 1 | 428.00 | 426.34 | 0.39% | :white_check_mark: |
| cadene-resnext101_1 | 1 | 360.58 | 368.30 | -2.10% | :white_check_mark: |
| onnx-taau-downsample | 1 | 397.72 | 397.43 | 0.07% | :white_check_mark: |
| dlrm-criteoterabyte | 1 | 31.92 | 31.91 | 0.02% | :white_check_mark: |
| dlrm-criteoterabyte_fp16 | 1 | 51.01 | 51.01 | 0.00% | :white_check_mark: |
| agentmodel | 1 | 9,602.92 | 9,938.09 | -3.37% | :red_circle: |
| unet_fp16 | 2 | 59.07 | 59.02 | 0.09% | :white_check_mark: |
| resnet50v1_fp16 | 1 | 995.13 | 1,093.99 | -9.04% | :red_circle: |
| resnet50v1_int8 | 1 | 988.17 | 992.35 | -0.42% | :white_check_mark: |
| bert_base_cased_fp16 | 64 | 1,099.24 | 1,099.78 | -0.05% | :white_check_mark: |
| bert_large_uncased_fp16 | 32 | 343.76 | 343.93 | -0.05% | :white_check_mark: |
| bert_large_fp16 | 1 | 198.07 | 198.07 | 0.00% | :white_check_mark: |
| distilgpt2_fp16 | 16 | 2,074.98 | 2,075.97 | -0.05% | :white_check_mark: |
| yolov5s | 1 | 587.98 | 588.16 | -0.03% | :white_check_mark: |
| tinyllama | 1 | 43.80 | 43.81 | -0.02% | :white_check_mark: |
| vicuna-fastchat | 1 | 44.89 | 45.07 | -0.40% | :white_check_mark: |
| whisper-tiny-encoder | 1 | 409.82 | 409.90 | -0.02% | :white_check_mark: |
| whisper-tiny-decoder | 1 | 413.75 | 414.63 | -0.21% | :white_check_mark: |
| llama2_7b | 1 | 19.11 | 19.13 | -0.10% | :white_check_mark: |
| qwen1.5-7b | 1 | 23.42 | 23.43 | -0.04% | :white_check_mark: |
| phi3-3.8b | 1 | 26.63 | 26.59 | 0.13% | :white_check_mark: |
| mask-rcnn | 1 | 12.16 | 12.16 | -0.02% | :white_check_mark: |
| llama3-8b | 1 | 21.68 | 21.65 | 0.14% | :white_check_mark: |
| whisper-large-encoder | 1 | 10.17 | 10.17 | -0.03% | :white_check_mark: |
| whisper-large-decoder | 1 | 99.58 | 99.72 | -0.13% | :white_check_mark: |
| mistral-7b | 1 | 23.64 | 23.64 | -0.03% | :white_check_mark: |
| FLUX.1-schnell | 1 | 722.64 | 728.97 | -0.87% | :white_check_mark: |
| nan | nan | nan | nan | nan% | :x: |
This build is not recommended to merge :red_circle:
:x:bert-mrpc-tf: ERROR - check error output
2025-09-23 19:50:22.358918: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 359, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 306, in main
graph = load_tf_graph(model_name)
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 300, in load_tf_graph
graph_def.ParseFromString(f.read())
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/lib/io/file_io.py", line 116, in read
self._preread_check()
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/lib/io/file_io.py", line 77, in _preread_check
self._read_buf = _pywrap_file_io.BufferedInputStream(
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme '[local]' not implemented (file: '/new-saved-models/tf-misc/bert_mrpc1.pb'):red_circle:bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output
:red_circle:mask-rcnn: FAILED: MIGraphX is not within tolerance - check verbose output