POS Tagging

[Killthebug]

Hi @kolloldas, great job with the transformer. I was using your model to run a few basic experiments on sequence labeling and after completing chunking and NER, wanted to move ahead with POS tagging. From what I understand, I'll need to create a new pos.py file in torchnlp/data/? Could you give me a heads up on if I'm on the right track or there's an easier work-around?

Thanks! :D

[kolloldas]

Thanks @Killthebug. You are on the right track! You can duplicate ner.py at the same level and rename symbols to pos appropriately. Only conll2003 dataset supports POS as of now so you should remove any references to nyt_ingredients_ner_dataset. You probably want to turn off use_iob_metrics in the call to train in the last line of the file as it isn't useful for POS. Hope this works! Let me know if you face any problems.