fast-bert icon indicating copy to clipboard operation
fast-bert copied to clipboard

Published 20 hours ago • verified publisher icon utterworks

Using multiple training instances in AWS Sagemaker.

Open nectario opened this issue 4 years ago • 4 comments

Is it possible to speedup BERT training by using multiple training instances?

nectario avatar Apr 23 '20 12:04 nectario

You can use p3.8xlarge and above for parallel processing across multiple gpus

kaushaltrivedi avatar Apr 23 '20 17:04 kaushaltrivedi

You need to set multi_gpu flag to true

kaushaltrivedi avatar Apr 23 '20 17:04 kaushaltrivedi

Thank you. Which config file do I set this at?

nectario avatar Apr 23 '20 18:04 nectario

@nectario You would set this in the initialization of BertDataBunch.

aaronbriel avatar May 08 '20 23:05 aaronbriel