torchrec icon indicating copy to clipboard operation
torchrec copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

Add unsharded module reference to sharded modules

Open aporialiao opened this issue 7 months ago • 5 comments

Summary: Adding a simple unsharded module reference to sharded modules. This will be used in Dynamic Sharding by DistributedModelParallel to reshard an already-sharded_module.

As DMP is created with only one-way relationship in mind, accessing the unsharded module type will help determine which sharder to use in 'resharding'.

Differential Revision: D73407830

aporialiao avatar Apr 22 '25 00:04 aporialiao