Tiny bert for arabic language

[ReySadeghi]

hi, I studied your article, but I have some questions if you could answer me.

1. what does the teacher bert means? it means the architecture of general bert?
2. we should train tiny bert from an existing english bert or it should be trained from scratch?
3. I want to make tiny bert for arabic language. up to now there is some arabic bert so I should use them as a teacher bert or I should train the bert for arabic from scratch? thanks

[chauncy-cc]

1. "Teacher bert" refers to bert base, which was proposed by Google.
2. Bert base is an existing english bert. We can use it to distill tinyBert.
3. I think you can use it as a teacher bert.