xla icon indicating copy to clipboard operation
xla copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

How to use spmd to support hybrid shard data parallelism？

Open mars1248 opened this issue 7 months ago • 3 comments

❓ Questions and Help

Fsdp can be well expressed by spmd, but hsdp seems to be unable to be expressed. Is there any way to express hsdp in spmd?

mars1248 avatar Jul 02 '24 08:07 mars1248