tokenizers icon indicating copy to clipboard operation
tokenizers copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Unable to Load Custom GPT2 Tokenizer - " data did not match any variant of untagged enum ModelWrapper at line 1 column 3193814" Error

Open maghwa opened this issue 7 months ago • 7 comments

I am encountering an issue when trying to load a custom merged GPT2 tokenizer using GPT2TokenizerFast. Despite ensuring that the tokenizer.json file is correctly formatted, I receive the following error: data did not match any variant of untagged enum ModelWrapper at line 1 column 3193814

tokenizer-2.json

maghwa avatar Jul 05 '24 14:07 maghwa