Pretrained-Language-Model
Pretrained-Language-Model copied to clipboard
Tiny bert for arabic language
hi, I studied your article, but I have some questions if you could answer me.
- what does the teacher bert means? it means the architecture of general bert?
- we should train tiny bert from an existing english bert or it should be trained from scratch?
- I want to make tiny bert for arabic language. up to now there is some arabic bert so I should use them as a teacher bert or I should train the bert for arabic from scratch? thanks
- "Teacher bert" refers to bert base, which was proposed by Google.
- Bert base is an existing english bert. We can use it to distill tinyBert.
- I think you can use it as a teacher bert.