AMDMIGraphX
AMDMIGraphX copied to clipboard
ROCmSoftwarePlatform
Cleanup CI to import Onnxruntime prebuild wheel
Move our the building of Onnxrt from all our CI and perform this action nightly/when there's an update we pull in for onnxrt
Potentially do the same for unit tests. Most likely we can leverage the MIGraphX Docker we build already to perform this
Removing builds save overhead on all CI by half an hour atleast. Removing unit tests cuts down another hour
Conclusion from this weeks (10/15/2023) meeting was to do the following
- [x] Build and tag new docker with wheel after integration is updated
- [x] Pull docker as part of CI run for unit tests and wheel -> may be able to get away with wheel
If we want to just store a wheel file/artifact that leaves
- Where do we store said wheel vs dockerhub?
- Unit tests where do we store those artifacts?
- Frequency of Unit tests (nightly/weekly?)
The plan is at worse, save half an hour every CI run + overhead, or at best an entire hour (build/unit tests done out of sync from regular CI)
@causten @pfultz2 let me know if I've missed anything on this. Don't want to lose sight of this once since it scales our team's velocity to get stuff in.
Waiting on devops for this one. Have changes in a PR from the antifactory build for now to get through CI. Hoping to close this out at the start of the new year
Moving this sprint ahead. Need to find place to pull Onnxrt build folder which contains test binaries
Using a docker image to handle the test binaries. This is similar to the change in DLM where we need a default container that's publicly hosted.
https://hub.docker.com/r/rocm/onnxruntime/tags I've added rocm/onnxruntime:latest_test which should be used for our testing efforts as well as tags rocm6.xx_other_components we should use for CI. I'll create an MIGraphX_CI_test tag which we can hang back if needed as we roll our ROCm release forward while still keeping coverage of Onnxruntime in our build environment without requiring us to constantly rebuild Onnxruntime for every PR and leave that for our weekly integration github action runs
Marking as revisit as we've pushed off build/maintaining ROCm 6.1 + Onnxruntime wheels to devops team.
Can't get UT zip from a stable place. Closing this out but ROCm 6.2 CI update allows us to just build MIGraphX EP which is a smaller overhead (~5 min) and then we can use the unit tests directly after that.