adding same document multiple times should reject that in search level

[munjalmukul]

🚀 The feature

How Its Working Currently Right now If I am adding same document twice its first searching memories and then updating, adding or deleting memory.

How It Should be There should be a layer at embedding search level so it reject the duplicate document to save llm cost.

Motivation, pitch

There are times when duplicate documents are uploaded by user so I dont want to send that document to llm so cost can be saved.

[NavyaAlapati13]

@Dev-Khant I am working on this enhancement, can you assign me this this issue? Looking forward to share the PR soon.

[Dev-Khant]

Sure @NavyaAlapati13, Thanks for working on it.

[deshraj]

Closing the issue since this is fixed now.