[Question]: How to merge output from flair with NER model

[B0rner]

Question

Hey,

I'm fusing flair with the ner-english-ontonotes-large model to determine entities in text, which is working really great. Further processing of these NER results becomes difficult when texts contain certain entities differently. For example, If I have a news about the greatest duck of Duckburg: Donald Duck, like this:

"Donald Duck is the famous person from Duckburg. Donald lives there with his family"

Flair/NLP will generate the 2 person entries: "Donald Duck" and "Donald".

I know, this is probably not a flair specific question, but is there a way, to merge/find the connection between "Donald Duck" and "Donald"?

The use case is to collect for example all the persons in a text and it is sub-optimal, if the output handles "Donald Duck" and "Donald" as different persons.

On the other hand, the model is great to recognize when the same word does not belong to the same entity, like Hamburger. The model exactly "knows" if the is the GPE, NORP or a PRODUCT.

What I need is the reverse case: different words that mean the same thing.

Any idea how to handle/merge this?

[helpmefindaname]

Hi @B0rner ,

If I understand your question correctly, there would be two approaches:

• Entity Linking works, if all mentions can be mapped to a known database (e.g. wikipedia-id), then the entity-linking would link both "Donald Duck" as well as "Donald" (due to the given context) to the same id. Notice: this won't work, if you are expecting an open vocabulary.
• A simplified case of Coreference Resolution which basically links Entities/Mentions together, if they reference the same person. Usually this is implemented as a relation-extraction model .

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[B0rner]

@helpmefindaname Thanks for giving some hints. Entity Linking sounds interesting, because I'm looking to merge the real famous entities. Your links looks like it points to a concept only.

But in general, the idea is to search the entity, like "Donald" with a bit of a text-kontext, like the sentence, where the entities come from? "Donald comes from Duckburg." - followed by crossing the fingers, that the search will return the id from Donald Duck?