superdu12138 comments

Repositories
Issues
Comments

Results 2 comments of


                                            superdu12138

How is the number of BERT model parameters calculated?

通常情况 transformer 模型有很多参数需要训练。譬如 BERT BASE 模型: L=12, H=768, A=12, 需要训练的模型参数总数是 12 * 768 * 12 = 110M https://zhuanlan.zhihu.com/p/51413773

How is the number of BERT model parameters calculated?

> 我对110M参数感到有点困惑。它是如何计算的？通常情况变压器模型有很多参数需要训练。譬如BERT BASE模型：L = 12，H = 768，A = 12，需要训练的模型参数总数是12 * 768 * 12 = 110M https://zhuanlan.zhihu.com/p/51413773