onnxruntime
onnxruntime copied to clipboard
microsoft
Add CANN EP
Description: This PR adds Ascend CANN execution provider support.
Motivation and Context
- Why is this change required? What problem does it solve?
As the info shown in the issue. CANN is the API layer for Ascend processor. Add CANN EP can allow user run onnx model on Ascend hardware via onnxruntime
The detail change:
- Added CANN EP framework.
- Added the basic operators to support ResNet model.
- Added C/C++、Python API support
- If it fixes an open issue, please link to the issue here. https://github.com/microsoft/onnxruntime/issues/11477
Author: lijiawei [email protected] wangxiyuan [email protected]
@skottmckay @edgchen1 Hi, could you please take a look at this PR? BIG thanks. I'm eager to know what should I do to push forward this code.
@skottmckay @edgchen1 Hi, could you please take a look at this PR? BIG thanks. I'm eager to know what should I do to push forward this code.
To other reviewers: I reviewed the shared provider library/API aspects of it, I didn't look in great detail into the provider code itself.
@souptc @RyanUnderhill @skottmckay @edgchen1 Hi, sorry to @ you again. The new commit is ready for review. All the GPU related code has been removed. Big thanks for your help.
@souptc @RyanUnderhill @skottmckay @edgchen1 Hi, sorry to @ you again. could you please take a look at this PR？
@souptc @RyanUnderhill @skottmckay @edgchen1 Hi, sorry to @ you again. could you please take a look at this PR？Thanks a lot.
![azure-pipelines[bot] avatar](https://avatars.githubusercontent.com/in/9426?v=4 "Published by a pub.dev verified publisher"){kind=small}
/azp run Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline
/azp run Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, onnxruntime-python-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed
A couple more follow-ups.
- you'll need to fix a build break due to GetCapability() signature change caused by https://github.com/microsoft/onnxruntime/pull/12791
- add some documentation. our docs are in the gh-pages branch. see https://github.com/microsoft/onnxruntime/tree/gh-pages/docs you'll need to create a separate docs PR and make appropriate modifications to build/eps.md and add a execution-providers/CANN-ExecutionProvider.md
- if you have some code examples/jupyter notebooks etc. they can go in https://github.com/microsoft/onnxruntime-inference-examples
A couple more follow-ups.
- you'll need to fix a build break due to GetCapability() signature change caused by Update kernel matching logic: decouple from op schemas and remove kernel def hashes #12791
- add some documentation. our docs are in the gh-pages branch. see https://github.com/microsoft/onnxruntime/tree/gh-pages/docs you'll need to create a separate docs PR and make appropriate modifications to build/eps.md and add a execution-providers/CANN-ExecutionProvider.md
- if you have some code examples/jupyter notebooks etc. they can go in https://github.com/microsoft/onnxruntime-inference-examples
Thanks for your suggestion. It's really helpful. We'll refresh the PR ASAP.
Docs、Example、More operator、 Performance Optimizing and so on are on our TODO list. We'll keep contributing in the future.
/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline
/azp run Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, onnxruntime-python-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed