crfsuite icon indicating copy to clipboard operation
crfsuite copied to clipboard

Restrict tags for each item

Open y-bo opened this issue 8 years ago • 4 comments

Is there a way to restrict possible set of tags for each item? For example, I want to do Morphological Disambiguation, so for each word there is a small set of possible tags (from dictionary), as opposed to all possible tags for all words.

y-bo avatar Oct 05 '16 13:10 y-bo

Perhaps I don't understand your problem but to me CRF is about what tags are possible for each item given some training set. In other words, your model learns what are the most likely tags at each item in the sequence. Finally, a priori one does not generally know at which element are the likely tags.

usptact avatar Oct 06 '16 02:10 usptact

@usptact the reason to restrict a number of tags is efficiency: you may have 1000 tags in your tag set, but only 5 possible tags for an item according to a dictionary. Time complexity is O(N^2) regarding to a number of tags, so the effect can be pretty large.

kmike avatar Oct 06 '16 07:10 kmike

@kmike Thanks for the clarification! I see the point now. For this to work, if I understand correctly, one must know at which items to restrict the possible tag set.

usptact avatar Oct 06 '16 08:10 usptact

@usptact Yeah, in my case for each item there is a set of features which contains possible tag set.

y-bo avatar Oct 10 '16 09:10 y-bo