openai_trtllm
openai_trtllm copied to clipboard
llama 3 tokenizer no longer works - updated eos token
The official llama 3 70b instruct repo has updated the eos token
"eos_token": "<|eot_id|>",
Yet when using this library and using that eos token, no output is outputted because it used the old eos token.
Suggesting to fix this @npuichigo