Bert_Chinese_Ner_pytorch icon indicating copy to clipboard operation
Bert_Chinese_Ner_pytorch copied to clipboard

label处理没有与input对齐

Open ALLinLLM opened this issue 5 years ago • 0 comments

/home/csx/remote_workdir/Bert_people_paper_2014_pytorch/preprocessing/data_processor.py line 244-247 ## Notes: label_id中不包括[CLS]和[SEP] label_id = [label_map[l] for l in labels] label_padding = [-1] * (max_seq_length-len(label_id)) label_id += label_padding 考虑到input第一个字符是CLS,最后一个字符是SEP,label在padding时应该和input对齐,开头结尾分别是CLS,SEP,置为-1 label_id = [-1]+label_id +[-1] * (max_seq_length-len(label_id)-1)

ALLinLLM avatar Jan 16 '20 11:01 ALLinLLM