MaticsL comments

Repositories
Issues
Comments

Results 1 comments of


                                            MaticsL

OverflowError: out of range integral type conversion attempted while running python qlora.py

same issue **update** Change model.config.pad_token_id to 0 should fix this problem but may harm to training. ```python tokenizer.add_special_tokens( { "eos_token": tokenizer.convert_ids_to_tokens(model.config.eos_token_id), "bos_token": tokenizer.convert_ids_to_tokens(model.config.bos_token_id), "unk_token": tokenizer.convert_ids_to_tokens(0), } ) ```