Pretrain large BERT model with RetVec - code and colab tutorial

Open MarinaZhang opened this issue 3 years ago • 2 comments

Oct 26 '22 20:10 MarinaZhang

Hi @MarinaZhang ,

many thanks for releasing the code for the RetVec paper! I would be interesting in pretraining a BERT model with RetVec so I just wanted to ask, if you e.g. have already a demo code and colab tutorial available :thinking:

Many thanks!

Nov 30 '23 13:11 stefan-it

Hello! are you interested in pre-training a BERT model on your own dataset or just finetuning a pre-trained one? We don't have a demo colab available yet but hopefully soon, I'll reopen this issue to work on it

Nov 30 '23 18:11 MarinaZhang