FastChat
FastChat copied to clipboard
What's the minus 8 in generate_stream function for max_src_len??
Hello!
I would like to ask about the meaning of this line: https://github.com/lm-sys/FastChat/blob/a26db3c814889035d92c8ae80d6defbd7381ee55/fastchat/serve/inference.py#L189
max_new_tokens
is for the space for the new generation but what's the 8
for?
Thanks in advance for your help : )