Zixuan Jiang

icon company Google Core ML icon location Shardy, SPMD, Horizontal Scaling

Results 1 comments of Zixuan Jiang

Shardy support

V2 HloSharding represents the device list in a compact way. The relation between V1 and V2 is ``` numpy.arange(num_devices).reshape(reshape_dims).transpose(transpose_perm).reshape(num_devices) ``` For example, the following two are equivalent. ``` V1: devices=[2,2,4]0,8,4,12,1,9,5,13,2,10,6,14,3,11,7,15...