albert_zh
albert_zh copied to clipboard
pretrained english models for tiny Albert?
The recent official release did not contain any info about the Tiny Bert published here,
So i was wondering if there are pretrained english versions of this tiny model (all I could find are Chinese), which would certainly be in demand. I see there are instructions for pretraining in english, are these up to date?
Tiny BERT is, in fact quite impressively tiny, and I would like to create an english version if possible.
Thank you