Daniil Cherniavskii

icon location Amsterdam icon location PhD student @ University of Amsterdam Previously researcher @ AIRI

Results 1 issues of Daniil Cherniavskii

TinyBERT for masked LM

Hi! I've been trying to measure MLM perplexity for TinyBERT model (in particular, [tinybert6l](https://huggingface.co/huawei-noah/TinyBERT_General_6L_768D)), and I keep getting inconsistent results. Looks like the MLM head for TinyBERT is not loaded...