snorkel Providing negative classification for multi-class classification.

Providing negative classification for multi-class classification.

Open mhigginslp opened this issue 6 years ago • 5 comments

In many cases, it would be useful to say when a particular instance is not a member of a particular class. For instance, an LF might want to say if a text contains the word w, then its corresponding class is not X. In the binary case, the suggested label would just be -1 but I don't see how that would work in the multi-class setting.

Feb 21 '19 16:02 mhigginslp

Hi @mhigginslp great suggestion! We don't have this feature right now, but certainly want to put it in as a PR at some point soon.

Basically, in Snorkel we learn the LF accuracies (in the generative model) by observing the agreements and disagreements between the LFs- so nothing fundamental in our approach changes here. More generally, at some point we want to let LFs output probability distributions over the possible labels, which captures your setting (0 prob. on class X, uniform over the other ones). Again, no fundamental changes to the approach (at least in the basic way of doing this).