FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

What's the minus 8 in generate_stream function for max_src_len??

Open gabinguo opened this issue 1 year ago • 0 comments

Hello!

I would like to ask about the meaning of this line: https://github.com/lm-sys/FastChat/blob/a26db3c814889035d92c8ae80d6defbd7381ee55/fastchat/serve/inference.py#L189

max_new_tokens is for the space for the new generation but what's the 8 for?

Thanks in advance for your help : )

gabinguo avatar May 06 '23 20:05 gabinguo