openai_trtllm icon indicating copy to clipboard operation
openai_trtllm copied to clipboard

llama 3 tokenizer no longer works - updated eos token

Open avianion opened this issue 1 month ago • 2 comments

The official llama 3 70b instruct repo has updated the eos token

"eos_token": "<|eot_id|>",

Yet when using this library and using that eos token, no output is outputted because it used the old eos token.

Suggesting to fix this @npuichigo

avianion avatar May 13 '24 13:05 avianion