Chinese-LLaMA-Alpaca-2 icon indicating copy to clipboard operation
Chinese-LLaMA-Alpaca-2 copied to clipboard

Published 20 hours ago • verified publisher icon ymcui

Added control to tokenizer for pad_token

Open yusufcakmakk opened this issue 7 months ago • 1 comments

I realized that there is no control for padding token when using sft trainer. So we can control it for custom tokenizers.

yusufcakmakk avatar Nov 22 '23 11:11 yusufcakmakk