tango
tango copied to clipboard
Add integration for native PyTorch FSDP
https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/