Why setting LLaMa3's padding direction to "right"?

[ROIM1998]

Hi! Really appreciate your great work.

I'm a bit confused of the padding_direction being set in LLaMA3's tokenizer.json file. As said in the comments, this is used in the model's repack function. Since LLaMA3 is a autoregressive model, why did you choose to pad the embeddings and placeholder labels to the right instead of left?

Also, padding to right raises an issue where the end of the input prompt is difficult to be identified during inference. If I want to finetune the model on my own dataset, will it still work if I change the padding side from right to left? Thanks!

[yaolug]

Could you provide a link to the code of the padding behavior that you are asking about? Thanks.

[JBurtn]

I'm curious too. I think @ROIM1998 is talking about here: https://github.com/NVlabs/VILA/blob/da98f3b98191540bbc52a9feea7102e1268b9c4c/llava/model/language_model/builder.py#L87-L108 There is also "tokenizer_padding_side": "right" from the config.json