maxtext icon indicating copy to clipboard operation
maxtext copied to clipboard

Support `allow_split_physical_axes` for multislice hybrid meshes such as FSDP on ICI and DCN.

Open Obliviour opened this issue 4 months ago • 0 comments

allow_split_physical_axes is only supported for device meshes atm but we also should support this for hybrid meshes. This is useful when we want to use FSDP across DCN and ICI + split the physical axis.

Obliviour avatar Oct 08 '24 22:10 Obliviour