maxtext icon indicating copy to clipboard operation
maxtext copied to clipboard
Published 2 days ago • verified publisher icon AI-Hypercomputer

[WIP] Support DiLoCo training

Open jonb377 opened this issue 1 year ago • 0 comments

DiLoCo training paper: https://arxiv.org/pdf/2311.08105

Adds preliminary support for DiLoCo training through DrJax.

jonb377 avatar Nov 07 '24 23:11 jonb377