llama-lora-fine-tuning icon indicating copy to clipboard operation
llama-lora-fine-tuning copied to clipboard

Run deepspeed fastchat/train/train_lora.py error. Padding Error

Open cyrilakafia opened this issue 2 years ago • 1 comments

I get this error when I run deepspeed fastchat/train/train_lora.py

ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as pad_token (tokenizer.pad_token = tokenizer.eos_token e.g.) or add a new pad token via tokenizer.add_special_tokens({'pad_token': '[PAD]'}).

Upgrading fschat did not help.

cyrilakafia avatar Aug 01 '23 09:08 cyrilakafia

add: tokenizer.add_special_tokens( {

            "pad_token": "<PAD>",
        }
    )

under this line:tokenizer.pad_token = tokenizer.unk_token

little51 avatar Aug 01 '23 11:08 little51