dkpro-core
dkpro-core copied to clipboard
Warn/avoid token/annotation duplication/overwriting
(Feature request originally posted on google.code forum at : https://groups.google.com/forum/#!topic/dkpro-core-developers/DEIpxBxQAzQ )
Running a reader and a segmenter in the same pipeline entails duplication of Token instances by the consumer.
Similarly an annotator may overwrite annotations created by the reader if they belong to the same type.
It would be nice to have a generic way to at least warn users, if not avoid, that a pipeline so combined may suffer from duplication/overwriting.