QinZhun

> Hi, I meet the same problem as you. I process the corpus with the pytorch version implementation by huggingface (https://github.com/huggingface/pytorch-pretrained-BERT/tree/master/examples/lm_finetuning). They claim that it is implemented according to "create_pretraining_data.py"...