Macaw-LLM icon indicating copy to clipboard operation
Macaw-LLM copied to clipboard

Published 20 hours ago • verified publisher icon lyuchenyang

Which tokenizer are you using?

Open sbkim052 opened this issue 9 months ago • 0 comments

Which tokenizer are you using? tokenizer = AutoTokenizer.from_pretrained('trained_models/llama_tokenizer') This seems to not work.

sbkim052 avatar May 22 '24 16:05 sbkim052