Zac comments

Repositories
Issues
Comments

Results 2 comments of

Zac

NLTK Sentence tokenizer does not tokenize properly if there exists 'e.g.' or 'i.e.' in sentence.

This is not reproducible as described. The training algorithm will actually catch cases like e.g. if they are followed by a comma. However, the bug behavior does occur without the...

NLTK Sentence tokenizer does not tokenize properly if there exists 'e.g.' or 'i.e.' in sentence.

The README included with the tokenizer says English.pickle was trained on PTB, and the sample 5% corpus contains zero instances of "e.g." and "i.e." so that makes sense. I was...