dkpro-core icon indicating copy to clipboard operation
dkpro-core copied to clipboard

Warn/avoid token/annotation duplication/overwriting

Open ismasensei opened this issue 9 years ago • 0 comments

(Feature request originally posted on google.code forum at : https://groups.google.com/forum/#!topic/dkpro-core-developers/DEIpxBxQAzQ )

Running a reader and a segmenter in the same pipeline entails duplication of Token instances by the consumer.

Similarly an annotator may overwrite annotations created by the reader if they belong to the same type.

It would be nice to have a generic way to at least warn users, if not avoid, that a pipeline so combined may suffer from duplication/overwriting.

ismasensei avatar Feb 02 '16 14:02 ismasensei