blog icon indicating copy to clipboard operation
blog copied to clipboard
verified publisher icon huggingface

There are 39000 words in my vocab.txt, but the model vocab_size is set to 50000. In this case, how does the embedded weight of 50000 * 768 output correspond to my 39000 words, is simply the first 39000? thanks

Open cc360 opened this issue 4 years ago • 1 comments

cc360 avatar Aug 13 '21 10:08 cc360

By the way, does the order of the output words' embedding correspond to the order of words in vocab.txt? thanks

cc360 avatar Aug 13 '21 10:08 cc360