inception-external-recommender icon indicating copy to clipboard operation
inception-external-recommender copied to clipboard

Span-tagging example code [feature request]

Open pseudomonas opened this issue 3 years ago • 7 comments

There are as far as I can tell no internal recommenders for spans (the extents of the spans rather than the labels attached to the spans); so any guidance on how to write an external trainable recommender for generic nested spans would be really welcome.

pseudomonas avatar Sep 18 '22 18:09 pseudomonas

There are different internal recommenders for different kinds of spans in INCEpTION, e.g.:

  • Sentence Classifier (OpenNLP Document Categorizer) if the span is to be an entire sentence
  • Token Sequence Classifier (OpenNLP POS) if every token is to be assigned a label
  • Multi-Token Sequence Classifier (OpenNLP NER) if the length of the span can vary (1+ tokens)

The label of the recommendation can be empty, but it is always required that a feature exists to which a label could be assigned. There is currently no way to define a recommender for a layer that has no features.

reckart avatar Sep 18 '22 18:09 reckart

Hmm. I'm aiming to do nested spans that have similar kinds of properties to phrases. I don't think any of those classifiers do that? I think what I probably need is something like a shift-reduce oracle or something vaguely of that flavour. But I'm far from an expert at this.

pseudomonas avatar Sep 18 '22 18:09 pseudomonas

A recommender must be assigned to a string feature and produces suggestions that have a begin, end, label (optional), score (optional) and description (optional). If you need to produce suggestions for multiple features of the same layer, you'd currently need to set up multiple recommenders, one for each feature.

reckart avatar Sep 18 '22 18:09 reckart

ah, so arbitrary levels of nesting aren't possible, because you need one recommender per level?

pseudomonas avatar Sep 18 '22 18:09 pseudomonas

You should be able to create overlapping suggestions if you needed that.

reckart avatar Sep 18 '22 19:09 reckart

Mind that the layer would need to be configured to allow for overlapping if you did that.

reckart avatar Sep 18 '22 19:09 reckart

Well, I'll give it a shot. Unless I go for a dependency grammar…

pseudomonas avatar Sep 18 '22 19:09 pseudomonas