Linly icon indicating copy to clipboard operation
Linly copied to clipboard

Published 20 hours ago • verified publisher icon CVI-SZU

'NoneType' object cannot be interpreted as an integer

Open tanklandry opened this issue 2 years ago • 2 comments

TencentPretrain/tencentpretrain/utils/dataloader.py", line 187, in iter yield torch.LongTensor(src),
TypeError: 'NoneType' object cannot be interpreted as an integer 请问这个是什么问题导致的，貌似预处理后的文件也没问题

tanklandry avatar Apr 13 '23 07:04 tanklandry

修改 utils/constants.py 文件L4，将 special_tokens_map.json 改为 llama_special_tokens_map.json

ydli-ai avatar Apr 13 '23 10:04 ydli-ai

修改后依旧报错 self.vocab.get(MASK_TOKEN) 为None

wind91725 avatar Apr 13 '23 17:04 wind91725