pytorch
pytorch copied to clipboard
[ROCm] Add support for SymmetricMemory
With the latest IFU into rocm6.3_internal_testing branch, we pulled in SymmetricMemory code which is being used by intra node communication also. SymmetricMemory also introduces a new memory allocator called CUDASymmetricMemoryAllocator. intra_node_comm is now using this new allocator to manage memory.