character-bert icon indicating copy to clipboard operation
character-bert copied to clipboard

MLM head pre-trained weights of character-bert

Open alexlimh opened this issue 2 years ago • 0 comments

Hi,

I just found that the input embeddings of character-bert encoder are not tied with the output MLM weights. I would appreciate it if you could also release the pre-trained general-character-bert models with the MLM head pre-trained weights, as our group would like to explore further pre-training using character-bert.

Many Thanks

alexlimh avatar Jun 03 '22 16:06 alexlimh