torch.cuda.device() does not seem to have an effect

[c0redumb]

🐛 Describe the bug

Using torch.cuda.device() to select the GPU, and it has no effect. GPU0 is always used, expecting GPU selected to be used. On the other hand, torch.cuda.set_device() works well. PyTorch documentation advice using torch.cuda.device() over set_device().

Versions

Found when using ROCm PyTorch 6.0.2 official docker container on node with MI-60 GPUs. Other systems not tested.