VedantParikh

How much time did it take to run the further pre-training step?

1

@xuyige Time taken !python run_pretraining.py \ --input_file=./tmp/tf_examples.tfrecord \ --output_dir=./tmp/pretraining_output \ --do_train=True \ --do_eval=True \ --bert_config_file=./uncased_L-12_H-768_A-12/bert_config.json \ --init_checkpoint=./uncased_L-12_H-768_A-12/bert_model.ckpt \ --train_batch_size=32 \ --max_seq_length=128 \ --max_predictions_per_seq=20 \ --num_train_steps=100000 \ --num_warmup_steps=10000 \ --learning_rate=5e-5 \...