lingvo icon indicating copy to clipboard operation
lingvo copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

Any plans or timelines to release gshard code?

Open ngoyal2707 opened this issue 4 years ago • 3 comments

Great work on gshard, curious if there are any plans to open-source it and if so, timeline?

ngoyal2707 avatar Jul 02 '20 15:07 ngoyal2707

Yes, We are thinking about to use GPT-3 or GShard, and open source code and models is really important for us.

DanielTizon avatar Aug 05 '20 11:08 DanielTizon

Our code has been open sourced. The instructions to run gshard dense transformer on gcp tpus are described here: https://github.com/tensorflow/lingvo/tree/master/lingvo/tasks/lm

bignamehyp avatar Feb 19 '21 07:02 bignamehyp

Our code has been open sourced. The instructions to run gshard dense transformer on gcp tpus are described here: https://github.com/tensorflow/lingvo/tree/master/lingvo/tasks/lm

No GPU-supported implementation?

gbxu avatar Feb 22 '21 06:02 gbxu