CTranslate2 icon indicating copy to clipboard operation
CTranslate2 copied to clipboard

Published 20 hours ago • verified publisher icon OpenNMT

[Feature] CANN Backend support

Open 3manifold opened this issue 5 months ago • 0 comments

CANN (Compute Architecture of Neural Networks), developed by Huawei, is a heterogeneous computing architecture for AI scenarios. It provides multi-layer programming interfaces to help users quickly build AI applications and services based on the Ascend platform.

CANN backend in CTranslate2, enables running AI models on the Ascend NPU extending the existing CPU & CUDA workflows. One can find more on Ascend NPU and CANN library here.

Examples of projects that support CANN include ONNX Runtime & OpenCV.

Implementation as well as detailed description of tests, environment setup & benchmarks can be found in pull request https://github.com/OpenNMT/CTranslate2/pull/1606 .

Notes

  • In the context of the development of this feature we also submitted issue https://github.com/OpenNMT/CTranslate2/issues/1583 .
  • In case CANN Backend support has increased demand, a new Pull request/subproject will be introduced contributing the respective CI involving dedicated Ascend hardware.

3manifold avatar Jan 29 '24 11:01 3manifold