Aditya Malte

Gives near SOTA results for eval_accuracy(IMDB) result equal to 0.9512. with batch size 32 and max_seq=256(Increasing max_seq further throws OOM error). Training and evaluation combined get completed within an hour...

Hello @kimiyoung , I have synchronized the fork to reflect the changes that have happened to the source repository. Any updates from your side? Thank you Best Regards, Aditya Malte

Hello @kimiyoung , I have updated my repo again to reflect changes with your latest xlnet repo, To summarise: 1) run_classifier remains exactly the same except the added flags: use_colab(default=False)...

The above merge PR, is in order to update my repo with all the changes made in yours

An 8 GB GPU may be too small, and you should be able to try a batch size of max 4-8 while keeping the seq_len equal to 128, which would...

Try using my Colab TPU notebook, working perfectly even for large batch sizes and acceptable sequence length. https://github.com/aditya-malte/Colab-XLNet-FineTuning

Do you have a colab ready notebook for XLNet now @ymcui . Please share it, I'm working on integrating colab with XLNet too.

I now have a notebook that's perfectly running with BERT TPUs, will share it shortly

With Colab TPU Tested Hello @hana9090 , @CharlieBickerton , @SchenbergZY , I have made available a colab Notebook with modified repo at the following link: https://github.com/aditya-malte/Colab-XLNet-FineTuning Please star and share.

Please do :)