SA-Sent icon indicating copy to clipboard operation
SA-Sent copied to clipboard

reader.py

Open PrivateThink opened this issue 6 years ago • 0 comments

hi,my python environment is 3.5,so i use stanford_corenlp_pywrapper has error,i found that reader.py uses tokenizer.tokenize ,Can I replace tokenizer.tokenize (sent_str) with word_tokenize (sent_str)?

def tokenize(self, sent_str): sent_str = " ".join(sent_str.split("-")) sent_str = " ".join(sent_str.split("/")) sent_str = " ".join(sent_str.split("!")) return word_tokenize(sent_str) #return tokenizer.tokenize(sent_str)

I need your help ,thanks

PrivateThink avatar Apr 24 '18 03:04 PrivateThink