csclimber

Results 2 comments of


                                            csclimber

python run.py with data_root=content/datasets num_gpus=2 num_nodes=1 task_mlm_itm whole_word_masking=True step100k per_gpu_batchsize=64

Have you solved it? I have the same bug.

python run.py with data_root=content/datasets num_gpus=2 num_nodes=1 task_mlm_itm whole_word_masking=True step100k per_gpu_batchsize=64

I solved it bro！ Don't forget to set these variables! export MASTER_ADDR=$DIST_0_IP export MASTER_PORT=$DIST_0_PORT export NODE_RANK=$DIST_RANK