Warning of right-padding for llama_tokenizer.padding_side

Open franciszchen opened this issue 2 years ago • 1 comments

In MiniGPT4 forward(), the llama_tokenizer is set as padding_side="right", as

https://github.com/Vision-CAIR/MiniGPT-4/blob/3bd99950f0ebcbbc7ee7b54aa33f332feeccef09/minigpt4/models/mini_gpt4.py#L177

But when performing llama_model.generate(), this would cause the warning, as follows:

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.

I am wondering if this would cause problems affecting predictions, and how to set correctly for the generate() case. Thx~

Apr 24 '23 05:04 franciszchen

Hey @franciszchen, By setting self.llama_tokenizer.padding_side = "left" could be the solution right?

Apr 28 '23 04:04 anurag-b72