opencti icon indicating copy to clipboard operation
opencti copied to clipboard

Extract Entity with NLP: step1

Open nino-filigran opened this issue 1 year ago • 2 comments

Use case

Details needs to be assessed, but the goal of the feature is to be able to use NLP to extract entities from unstrcutred content.

nino-filigran avatar Jan 23 '24 10:01 nino-filigran

@Jipegien I think the best and easiest way for you to leverage NER (Named Entity Recognition) to extract entities and relations out of unstructured texts is to use AI integration (OpenAI or your custom trained Mistral model). I already use such an approach before ingestion. Before I used this approach I tried it with well-known python frameworks with medium good results. After moving to NER with an AI model (First tried OpenAI, then Mistral; stayed at mistral because OpenAI gets to expensive if you run this on random web scraped content) the results were pretty good. Hope it helps a little.

vexvec avatar Apr 21 '24 11:04 vexvec

Yes we are considering this possibility, but in a context of multiple users, it comes rapidly with huge costs and latencies. Results are often not so consistent over time. Thanks :)

Jipegien avatar Apr 22 '24 07:04 Jipegien