paraphrastic-representations-at-scale icon indicating copy to clipboard operation
paraphrastic-representations-at-scale copied to clipboard

Download the Punkt tokenizer if it is missing

Open jtbates opened this issue 3 years ago • 0 comments

The paranmt preprocessing script add_overlap_labels.py fails if the Punkt tokenizer hasn't been downloaded. Added a check for it.

jtbates avatar Jun 03 '21 05:06 jtbates