Further Pre-Training on the IMDB dataset

[addiu]

Dear Yige, thanks a lot for sharing the code! I was wondering if you could provide some more detail on "further pre-training" on the IMDB dataset, e.g. the hyperparameter settings for it. Or, is it possible to share the BERT model which did the LM pre-training on the IMDB dataset?

[xuyige]

thank you for your issue we have shown some hyperparameters settings in our paper (see section 5.2) for bert checkpoints after further-pretraining, we share a link in our README (see section Further Pre-Trained Checkpoints) after clicking the link, you can get the imdb-based checkpoint in the file pytorch_model_len128_imdb.bin