german-gpt2 icon indicating copy to clipboard operation
german-gpt2 copied to clipboard

Published 20 hours ago • verified publisher icon stefan-it

No Vocab

Open SchahinRohani opened this issue 2 years ago • 1 comments

Hello, i am trying to use the model in julia/transformers.jl lib from the HuggingFace dbmdz/german-gpt2 After getting a error when loading the model, i found that the main branch doesnt have a vocab file.

Shouldnt it have vocab file, or should i add a vocab file?

SchahinRohani avatar Sep 04 '22 20:09 SchahinRohani

Hi @Schahin-R ,

the vocab is stored in the tokenizer.json file that can be found here:

https://huggingface.co/dbmdz/german-gpt2/tree/main

(The model is using a Fast Tokenizer, so merges.txt and vocab.json are not available compared to the original GPT-2)

stefan-it avatar Dec 01 '22 21:12 stefan-it