awesome-gpt3 icon indicating copy to clipboard operation
awesome-gpt3 copied to clipboard

Published 20 hours ago • verified publisher icon elyase

GPT vs BERT, under same computation and data resource, which one is better for downstream tasks like GLUE?

Open guotong1988 opened this issue 4 years ago • 1 comments

Thank you very much.

guotong1988 avatar Sep 30 '20 02:09 guotong1988

感觉BERT之类的用auto encoder模式来预训练的模型更适合GLUE吧，GPT比较擅长text generation

EricLee8 avatar Oct 22 '20 09:10 EricLee8