Bump ci rocm6.2
Updating CI to ROCm 6.2 and other appropriate dockers
Still waiting info for torch sha-1 used dont see it on repo.radeaon under the usual manylinux link. Will let this run over weekend in the meantime.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 92.04%. Comparing base (
7c2fdf5) to head (0bb51fc). Report is 174 commits behind head on develop.
Additional details and impacted files
@@ Coverage Diff @@
## develop #3340 +/- ##
=========================================
Coverage 92.04% 92.04%
=========================================
Files 505 506 +1
Lines 20699 20872 +173
=========================================
+ Hits 19052 19212 +160
- Misses 1647 1660 +13
| Flag | Coverage Δ | |
|---|---|---|
92.04% <100.00%> (?) |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Looks good besides the CI failures.
Looks good besides the CI failures.
Yeah. something I'll have to deal with I think. @pfultz, should I be including some of your changes to the CPP checks from the 6.1.3 CI update PR in CMakeLists.txt https://github.com/ROCm/AMDMIGraphX/pull/3335/commits/9291c014b94981b37e1a2241ff4d0630d0af9342
Yeah. something I'll have to deal with I think. @pfultz, should I be including some of your changes to the CPP checks from the 6.1.3 CI update PR in CMakeLists.txt
There are lots of changes needed from there, but it seems like hip-config.cmake is broken in 6.2, so we probably need to do some patching of the files.
Confirmed items are upstream on repo.radeon finally for the Onnxruntime wheel
Rebased off #3335
Looks like somehow we're failing on an env variable due to something not setting hip_HIPCC_EXECUTABLE for this. Tryin to see if I modify the hip-clang docker if this will resolve the failures in tidy. I also updated the files the license check was complaining about. I'll check back in the morning
Odd, micimaster all passed but I dont see this reflected in PR. Looks like its just tidy/cpp check warnings/errors now?
The pass is from 20 days ago. I'll rekick. If you update the PR with a new commit it clears the Jenkins run information
Got past the hipcc errors and the hip-config by modifying hip-clang.cmake usind sed to set the HIP_PLATFORM to "Amd"
Now hitting this
-- Generating embedding library 'migraphx_kernels'
-- Performing Test HAS_HIP_LAMBDA_HOST_DEVICE
-- Performing Test HAS_HIP_LAMBDA_HOST_DEVICE - Failed
-- Build with rocMLIR::rockCompiler 2.0.0
-- MIGraphX is using HIP Clang
CMake Error at cmake/TargetFlags.cmake:42 (get_target_property):
get_target_property() called with non-existent target "hip::device".
Call Stack (most recent call first):
cmake/TargetFlags.cmake:61 (get_target_property2)
cmake/TargetFlags.cmake:100 (append_flags)
src/targets/gpu/CMakeLists.txt:250 (target_flags)
-- Hip compiler flags: " "
-- Looking for miopenHiddenSetConvolutionFindMode in MIOpen
CMake Error at /data/build/CMakeFiles/CMakeTmp/CMakeLists.txt:20 (add_executable):
Target "cmTC_cab74" links to target "hip::host" but the target was not
found. Perhaps a find_package() call is missing for an IMPORTED target, or
an ALIAS target is missing?
CMake Error at /usr/share/cmake-3.22/Modules/CheckLibraryExists.cmake:72 (try_compile):
Failed to generate test project build system.
Call Stack (most recent call first):
src/targets/gpu/CMakeLists.txt:291 (check_library_exists)
Before we close this out we should make issues for the following tidy warnings that I just stubbed out
- [ ] bug-prone-empty-catch
- [ ] performance-enum-size
The other readability ones may not need issues
- [ ] readability-redundant-member-init
- [ ] readability-avoid-nested-conditional-operator
| Test | Batch | Rate new 0bb51f |
Rate old 32a84a |
Diff | Compare |
|---|---|---|---|---|---|
| torchvision-resnet50 | 64 | 3,251.39 | 3,250.93 | 0.01% | :white_check_mark: |
| torchvision-resnet50_fp16 | 64 | 6,981.90 | 6,986.29 | -0.06% | :white_check_mark: |
| torchvision-densenet121 | 32 | 2,433.85 | 2,433.76 | 0.00% | :white_check_mark: |
| torchvision-densenet121_fp16 | 32 | 4,111.85 | 4,110.78 | 0.03% | :white_check_mark: |
| torchvision-inceptionv3 | 32 | 1,635.90 | 1,635.58 | 0.02% | :white_check_mark: |
| torchvision-inceptionv3_fp16 | 32 | 2,738.41 | 2,737.29 | 0.04% | :white_check_mark: |
| cadene-inceptionv4 | 16 | 776.74 | 775.61 | 0.15% | :white_check_mark: |
| cadene-resnext64x4 | 16 | 808.50 | 808.58 | -0.01% | :white_check_mark: |
| slim-mobilenet | 64 | 7,453.48 | 7,452.87 | 0.01% | :white_check_mark: |
| slim-nasnetalarge | 64 | 208.35 | 208.18 | 0.09% | :white_check_mark: |
| slim-resnet50v2 | 64 | 3,434.90 | 3,434.69 | 0.01% | :white_check_mark: |
| bert-mrpc-onnx | 8 | 1,150.56 | 1,150.44 | 0.01% | :white_check_mark: |
| bert-mrpc-tf | 1 | 310.83 | 311.37 | -0.17% | :white_check_mark: |
| pytorch-examples-wlang-gru | 1 | 428.30 | 426.16 | 0.50% | :white_check_mark: |
| pytorch-examples-wlang-lstm | 1 | 390.88 | 386.06 | 1.25% | :white_check_mark: |
| torchvision-resnet50_1 | 1 | 773.02 | 816.95 | -5.38% | :red_circle: |
| cadene-dpn92_1 | 1 | 397.30 | 401.62 | -1.08% | :white_check_mark: |
| cadene-resnext101_1 | 1 | 381.22 | 380.63 | 0.16% | :white_check_mark: |
| onnx-taau-downsample | 1 | 344.52 | 345.07 | -0.16% | :white_check_mark: |
| dlrm-criteoterabyte | 1 | 35.06 | 35.06 | -0.00% | :white_check_mark: |
| dlrm-criteoterabyte_fp16 | 1 | 58.05 | 58.01 | 0.07% | :white_check_mark: |
| agentmodel | 1 | 8,046.15 | 7,975.82 | 0.88% | :white_check_mark: |
| unet_fp16 | 2 | 58.03 | 58.07 | -0.07% | :white_check_mark: |
| resnet50v1_fp16 | 1 | 934.55 | 921.74 | 1.39% | :white_check_mark: |
| resnet50v1_int8 | 1 | 935.51 | 953.21 | -1.86% | :white_check_mark: |
| bert_base_cased_fp16 | 64 | 1,152.88 | 1,152.80 | 0.01% | :white_check_mark: |
| bert_large_uncased_fp16 | 32 | 355.81 | 355.84 | -0.01% | :white_check_mark: |
| bert_large_fp16 | 1 | 211.62 | 211.95 | -0.16% | :white_check_mark: |
| distilgpt2_fp16 | 16 | 2,162.07 | 2,157.57 | 0.21% | :white_check_mark: |
| yolov5s | 1 | 536.73 | 528.76 | 1.51% | :white_check_mark: |
| tinyllama | 1 | 43.42 | 43.37 | 0.11% | :white_check_mark: |
| vicuna-fastchat | 1 | 171.25 | 168.85 | 1.42% | :white_check_mark: |
| whisper-tiny-encoder | 1 | 414.85 | 418.42 | -0.85% | :white_check_mark: |
| whisper-tiny-decoder | 1 | 428.64 | 425.66 | 0.70% | :white_check_mark: |
This build is not recommended to merge :red_circle:
:red_circle:bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output