cdec icon indicating copy to clipboard operation
cdec copied to clipboard

Published 20 hours ago • verified publisher icon redpony

Tokenizer + tags

Open vchahun opened this issue 12 years ago • 0 comments

Tokenizing bitext with <p> tags fails:

echo "x ||| <p>x</p>" | ~/tools/cdec/corpus/tokenize-anything.sh
x |||<p> x</p>

Why these tags are in my corpus is another problem :tired_face:

vchahun avatar Jan 31 '13 05:01 vchahun