Missing eos_id=tokenizer.eos_id in the generate function call in generate/full.py

[wlsdnen]

trafficstars

I've noticed something odd in generate/full.py. When calling the generate function, it looks like we're missing the eos_id=tokenizer.eos_id. Right now, it's set to None, and this is causing a series of mysterious codes to appear during inference. It seems like adding the eos_id argument could be the fix here.

[lantiga]

Great catch @wlsdnen, do you want to send a PR or should I do that?

[wlsdnen]

Great catch @wlsdnen, do you want to send a PR or should I do that?

Thank you. I will quickly make the corrections and send a PR.

[lucasjinreal]

@wlsdnen I met same problem, English seems not have such issue, but other language has serious issue, please help fix it.