electra icon indicating copy to clipboard operation
electra copied to clipboard

Published 20 hours ago • verified publisher icon google-research

pretrain with multigpu

Open 652994331 opened this issue 4 years ago • 3 comments

Hi, guys, can we use multi-gpus to pretrain electra-small or electra-base now. Anyone figured it out? thanks

652994331 avatar Nov 25 '20 00:11 652994331

Check out the NVIDIA's implementation. https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA

hiroshi-matsuda-rit avatar Jun 16 '21 10:06 hiroshi-matsuda-rit

Or this: https://github.com/richarddwang/electra_pytorch

PhilipMay avatar Jun 16 '21 12:06 PhilipMay

Or this: https://github.com/richarddwang/electra_pytorch

Hello，I tried electra_pytorch, but only one GPU was running. Have you ever run it with multigpu? How did you do that?

MrSworder avatar Oct 22 '21 10:10 MrSworder