albert icon indicating copy to clipboard operation
albert copied to clipboard

Published 20 hours ago • verified publisher icon google-research

do u have some contrastive datas between chinese Albert with other chinese pretraining models?

Open xiongma opened this issue 5 years ago • 3 comments

xiongma avatar Dec 31 '19 07:12 xiongma

Yes, you can find the comparison in the Chinese CLUE page (https://github.com/CLUEbenchmark/CLUE). Maybe it is because the way I trained it. The xxlarge model is sensitive to the downstream hyperparamters. You may need to run a search on the dev set to get good results.

Danny-Google avatar Jan 03 '20 17:01 Danny-Google

but in this website, the xxlarge or base model was trained by bright, which one is yours

xiongma avatar Jan 04 '20 12:01 xiongma

The xxlarge model of the first and second tables are trained by me. The xxlarge model is not very stable as there were some problems in training the xxlarge model. I am still fixing that, but others should be fine.

Danny-Google avatar Jan 06 '20 23:01 Danny-Google