alvations

@johnfarina Thanks for spotting that! The latest PR should #60 resolve the CJK issues. The Hindi one is a little more complicated, so leaving this PR open. ``` pip install...

Sorry for disappearing for quite a while. Do you have the input sentence that caused that error?

Need to hurry up with the docs... ETA: 20 May 19 Sorry for the delay for whoever is following this repo ;P

Bumping issue -_- ||| Please do this asap...

@mfoglio there are no pre-trained models in this library. They are purely rule-based ;P

Add tests to test `sacremoses` output against Moses' perl scripts.

Thanks @Patdue!!! That'll be very helpful!

Hmmm true. But the point is to keep the interface pythonic, but I agree it's confusing. Let me think of a better wording for the feature =)

The original script added that new hack that changed quite recently: https://github.com/moses-smt/mosesdecoder/pull/204 This difference isn't accounted for in sacremoses. And I'm really not sure whether we should or not.

Seems to be the behavior of default moses too =( ``` $ echo "yesterday ’s reception" | perl tokenizer.perl -l en | perl detokenizer.perl Detokenizer Version $Revision: 4134 $ Language:...