lit-llama icon indicating copy to clipboard operation
lit-llama copied to clipboard

Published 20 hours ago • verified publisher icon Lightning-AI

Use of left padding

Open saiajaym opened this issue 1 year ago • 2 comments

Left padding makes more sense for auto regressive models. Also HuggingFace's implementation uses left padding for tokenization.

saiajaym avatar Jun 01 '23 04:06 saiajaym

Hi, I don't think we can use left-padding for training. See #77. Left-padding makes sense for batched inference. Could you elaborate a bit more on where you are suggesting a change?

awaelchli avatar Jun 01 '23 17:06 awaelchli

Hey @saiajaym I just want to make sure there is no misunderstanding here, could you describe concretely what change needs to be made and where?

awaelchli avatar Jun 06 '23 11:06 awaelchli