megablocks icon indicating copy to clipboard operation
megablocks copied to clipboard

Published 20 hours ago • verified publisher icon stanford-futuredata

Unsharding scripts for megablocks models

Open mayank31398 opened this issue 1 year ago • 0 comments

The base Megatron-LM repo provides unsharding scripts for the models which can be used after training. I didn't find any such scripts in the repo. Would it be possible to provide the same?

mayank31398 avatar Jan 21 '24 22:01 mayank31398