Question about MLM of layoutlmv3

[a396198172]

Hi,

Thanks for sharing layoutlmv3. There is a question about MLM of layoutlmv3. Are all of the words masked replaced by [MASK] token, or there is a ratio between [MASK] token, randomly replaced token and unchanged token, just like BERT's MLM? Could you share the specific ratio about it, please?

Thanks.

Hi @HYPJUDY , if you have time, please help me about this question, thanks.

[HYPJUDY]

Hi, similar to BERT, our ratios are 80%, 10%, and 10% for masked tokens, random tokens, and unchanged tokens, respectively.