Arabert icon indicating copy to clipboard operation
Arabert copied to clipboard
verified publisher icon zaidalyafeai

Compute Resources

Open zaidalyafeai opened this issue 5 years ago • 2 comments

A collection of free TPU compute

zaidalyafeai avatar Feb 16 '20 02:02 zaidalyafeai

For this task, Colab and Kaggle can only help in data preparation (may be not Kaggle if output is limited to 5 GB). For actual training, more compute resources are needed and for sometime. For example, in the last Arabic BERT dataset iteration, the hdf5 files were around 21 GB. My suggestion is to see if an individual or entity/institution has access to compute (ex. free credit on Google cloud or in-house/rented infrastructure that can be used during some time periods) - provided the output stays free to community (contribution acknowledgement granted).

abedkhooli avatar Feb 18 '20 18:02 abedkhooli

I've got this from TensorFlow

"We’re happy to invite you to use up to 5 on-demand Cloud TPU v3 devices, 5 on-demand Cloud TPU v2 devices, and 100 preemptible Cloud TPU v2 devices for free for 30 days."

  • Each Cloud TPU v3 provides up to 420 teraflops of processing power and 128 GB of high-bandwidth memory.
  • Each Cloud TPU v2 provides up to 180 teraflops of processing power and 64 GB of high-bandwidth memory.

I think this is enough.

zaidalyafeai avatar Feb 18 '20 21:02 zaidalyafeai