ParlAI icon indicating copy to clipboard operation
ParlAI copied to clipboard
verified publisher icon facebookresearch

Pre-train r2c2 from scratch

Open robinsongh381 opened this issue 3 years ago • 1 comments

I would like to pre-train r2c2-base-400M/3B from scratch with reddit+robert+cc100en datasets on ParlAI. Is there any way / script to do that ?

robinsongh381 avatar Sep 22 '22 02:09 robinsongh381

You will need to look elsewhere for the data, as we do not provide it within ParlAI. The SeeKeR paper outlines the training hyperparamters

klshuster avatar Sep 22 '22 16:09 klshuster

Ok, thank you !

robinsongh381 avatar Sep 27 '22 14:09 robinsongh381