espoof

From TreebankWordDetokenizer, the detokenize adds/subtracts spaces to/from special characters

2

After using word_tokenize() on the following sample string: "Hello ; Hello World . Hello ; ;Hello;" I use detokenize to put the string back together. I receive the following: "Hello;...