Suharu

Errors when pre-training Bert on local GPU

1

I got this error "ImportError: cannot import name 'trace' from 'tensorflow.python.profiler" when runing [python run_pretraining.py]. My environment is anaconda, with tensorflow-gpu==1.15.0, python==3.9.7. I did't do something particular, just git clone...

VM and Disk size when traing Bert in google TPU

1

Hello. Thank you very much for sharing your experience. I also want to pretrain a bert with google v3-8, and I want to ask about how to choose the VM...