CTranslate2 icon indicating copy to clipboard operation
CTranslate2 copied to clipboard

Published 20 hours ago • verified publisher icon OpenNMT

DirectML Support

Open gdiaz384 opened this issue 5 months ago • 2 comments

Assume that I already followed Microsoft's instructions to Enable PyTorch with DirectML on Windows and the DirectML library loads correctly according to MS's example code.

If I wanted to use PyTorch + DirectML + Ctranslate2 + an AMD GPU, how would I do it? Does Ctranslate2 need any major updates? Shouldn't I just need to pass directml instead of cuda to device= as found in the quickstart?

What is the bottleneck to this not working right now?

Aside: MS's instructions for DirectML's Python library install pytorch==2.0 as of early 2024. MS's documentation suggests it only works with PyTorch 1.3, which seems to be incorrect? So, it should not be a matter of supporting PyTorch 1.x since DirectML runs on 2.x in practice.

gdiaz384 avatar Feb 04 '24 23:02 gdiaz384