named_entity_recognition icon indicating copy to clipboard operation
named_entity_recognition copied to clipboard

potential fix in build_corpus

Open mikelty opened this issue 3 years ago • 0 comments

I changed an if-else block to try-except block and it worked. Machine: windows10, python3.7 also i need another sklearn package after i installed requirements.txt i think this is due to a syntactical difference between bmes format and a windows file reader. idk.

 def build_corpus(split, make_vocab=True, data_dir="./ResumeNER"):                                                           """读取数据"""                                                                                                          assert split in ['train', 'dev', 'test']                                                                                                                                                                                                        word_lists = []                                                                                                         tag_lists = []                                                                                                          with open(join(data_dir, split+".char.bmes"), 'r', encoding='utf-8') as f:                                                  word_list = []                                                                                                          tag_list = []                                                                                                           for line in f.readlines():                                                                                                  try:                                                                                                                        word, tag = line.strip('\n').split()                                                                                    word_list.append(word)                                                                                                  tag_list.append(tag)                                                                                                except:                                                                                                                     word_lists.append(word_list)                                                                                            tag_lists.append(tag_list)                                                                                              word_list = []                                                                                                          tag_list = []

mikelty avatar Mar 13 '21 15:03 mikelty