Perplexity for Left Padded Models

[Lawhy]

Hi, while using the perplexity pipeline to compute the scores for left padded causal LMs (e.g., gemma-2b), I found the results are not the same if I changed the padding side to right. I would like to ask if we should always set padding_side to right when computing perplexity scores?