LLMZoo icon indicating copy to clipboard operation
LLMZoo copied to clipboard

Published 20 hours ago • verified publisher icon FreedomIntelligence

About the tokenizer

Open yuyq96 opened this issue 1 year ago • 2 comments

Why the tokenizer class specified in tokenizer config is BloomTokenizer? There is only BloomTokenizerFast in transformers implementation.

yuyq96 avatar Apr 19 '23 01:04 yuyq96