Pytorch-POS-Tagger icon indicating copy to clipboard operation
Pytorch-POS-Tagger copied to clipboard

Published 20 hours ago • verified publisher icon Shivanshu-Gupta

data preprocess

Open cswangjiawei opened this issue 6 years ago • 1 comments

Can you tell me how do you process the data into the format of sentences.txt and tags.txt? Thank you very much .

cswangjiawei avatar Oct 22 '18 11:10 cswangjiawei

sentences.txt consists of one sentence per line as a space separated list of tokens. Similarly, tags.txt consists of space separated lists of tags for corresponding sentence tokens. Once you have sentence.txt and tags.txt ready, you can use preprocess.sh to preprocess them for use by the model.

Shivanshu-Gupta avatar Oct 22 '18 13:10 Shivanshu-Gupta