AMDMIGraphX
AMDMIGraphX copied to clipboard
ROCmSoftwarePlatform
AMD's graph optimization engine.
Follow up from work from #2584 scoping Add support for MIGraphX into Turnkey ML. Leverage the existing TensorRT route used and allow us to run an inference with MIGraphX through...
- MIGraphX commit: [352dcea](https://github.com/ROCm/AMDMIGraphX/commit/352dcea2c6a03c495a6ba8667e19811bc5d1399b) - GPU: MI210 (gfx90a) - ROCm version: 5.7.0 Model(s) to reproduce: - https://github.com/gyulaz-htec/models/blob/migraphx_testing/text/machine_comprehension/bert-squad/model/bertsquad-12-int8.tar.gz Fails at: `/code/AMDMIGraphX/src/onnx/onnx_parser.cpp:419: parse_graph: Unknown operator: FusedMatMul` Note: More info at [ContribOperators.md/com.microsoft.FusedMatMul](https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#com.microsoft.FusedMatMul)
There is two parts for this: 1. Use dynamic shapes to handle the past sequence length. 2. Similar to dynamic batching, we can make two different submodule for each GQA...
An example case would be when we have a square image. Such that the height and width are the same but dynamic.
--batch should only be used when there is only 1 input and it is the first dimension. Otherwise error with a message directing the user to use input dimms The...
* Extend code in `pass_manager.cpp:run_passes(prog, root_mod, passes, trace)` to handle updating the cached shape from `op.compute_shape(shape_inputs, mod_inputs)` when `mod_inputs` shape changes. * Currently, the cached shape is not updated when...
We currently only have a C++ API example for using dynamic batch. We need to make an example using the Python API and dynamic batch.
* Both APIs need to be updated to allow for dynamic batch to be used with user allocated device memory * Related to #1720 and #1734