Fix shortlist pruning for CJK

[eu9ene]

I don't have a good understanding of why some lines are suddenly empty as a result of running "extract_lex".

There are just a few of them and the model trained ok so I assume it's fine.

@ZJaume did you run into similar issues? Or you don't use the shortlist in your pipeline?

closes #753