llama icon indicating copy to clipboard operation
llama copied to clipboard

Embedding shape / Vocab size

Open Lrakotoson opened this issue 1 year ago • 2 comments

Hello to all, Thank you for this work.

I guess anyone who had access to the model weights as well as the authors can answer my question. I may have missed it in the paper but it seems to me that there is no mention of the embedding shape or just the tokenizer vocabulary size.

Lrakotoson avatar Feb 27 '23 15:02 Lrakotoson

I have the same question here. I didn't access the weights yet. so I am asking if any one has.

WikiGenius avatar Feb 27 '23 22:02 WikiGenius

The vocab size is 32000.

JOHW85 avatar Mar 02 '23 04:03 JOHW85