alvations

Heads up on JESC, it's pretty noisy =)

@advpetc Thank you for reporting the issue. Could you explain it with a little more detail? It's a little unclear what changes you are proposing. - What is the input...

Yes, the `MosesTokenizer` output in NLTK doesn't correspond to the one from Moses, the NLTK output shouldn't be the expected behavior: ``` ~/mosesdecoder/scripts/tokenizer$ perl tokenizer.perl -l en Tokenizer Version 1.1...

Yes, removing the extra space in nonbreaking prefix for the `No #NUMERIC_ONLY#` line solves the problem. After removing the extra space: ``` >>> from nltk.tokenize.moses import MosesTokenizer moses =>>> moses...

A regression test of Moses vs NLTK implementation would be good to test all these kinks =) I've just checked the `nonbreaking_prefixes.en` from https://github.com/alvations/mosesdecoder/blob/master/scripts/share/nonbreaking_prefixes/nonbreaking_prefix.en#L103 there's a space there too. So...

The "non-words" raised by @juh2 should have been resolved in #49 ```python >>> from nltk.corpus import stopwords >>> deu_stops = stopwords.words('german') >>> 'unse' in deu_stops False >>> 'unsem' in deu_stops...

Sorry for missing this issue out. There shouldn't be any problem with the framenet_v15.zip now in the latest version of nltk and nltk_data. @slremy Are you still having an issue...

The UI is a little buggy. Try using `nltk.download('framenet_v15')`

This is because both `verbnet` and `verbnet3` has the same `id`: ``` nltk_data/packages/corpora$ cat verbnet.xml nltk_data/packages/corpora$ cat verbnet3.xml ``` The same identifier is causing the mismatch in the `nltk` code...

The different resources in `nltk_data` comes in different licenses. The licenses of the individual resources in `nltk_data` should be safe for re-distribution. It'll be great to package `nltk_data`, would it...