AttributeError: module 'torch.distributed' has no attribute '_all_gather_base'

[AungV]

https://github.com/NVIDIA/apex/blob/6a40a0ad9ff3d6ebea715cf28faf39792312acbf/apex/transformer/utils.py#L10

when i use FusedAdam in torch1.8, there is no all_gather_into_tensor or _all_gather_base in dir(torch.distributed).

[anber1]

me too, do u solve it?

[AungV]

me too, do u solve it?

use old version 😀

[anber1]

ok thanks

[toohappygirl]

me too, do u solve it?

use old version 😀

What the old version is?How to use the old version?

[AungV]

me too, do u solve it?

use old version 😀

What the old version is?How to use the old version?

find when this code be changed(added).

[zhiyuuu]

I uninstalled the current version and installed "22.04-dev" of apex, then solved the same issue. (torch 1.9.1, cuda 11.1)

[callanwu]

I uninstalled the current version and installed "22.04-dev" of apex, then solved the same issue. (torch 1.9.1, cuda 11.1)

Thank you very much, the solution works perfectly for me. For reference, my PyTorch version is pytorch=1.8.1=py3.8_cuda11.1_cudnn8.0.5_0.