semantic-kernel icon indicating copy to clipboard operation
semantic-kernel copied to clipboard

.Net: MongoDB memory store connector should use internal ObjectId as Id and a separate string metadata.key

Open dviry opened this issue 2 years ago • 2 comments

The current MongoDB memory store connector uses the MemoryEntryMetadata "key" as the MongoDB "id" - and duplicates it - once in the MemoryRecordMetadata which btw shows up as _id (because it's attributed with BsonId) and once in the MemoryEntry itself.

That is not only architecturally wrong (using a string id in a MongoDB has a lookup/storage performance hit, it should be ObjectId), but it also means you cannot have multiple MemoryEntries for the same referenced Key (which is the issue I am having) due to the Upsert method using the Key to identify the MongoDB entry.

I'd suggest separating the internal MongoDB unique MemoryEntry.ID (should be of type ObjectId) - from the MemoryEntryMetadata.Key (should remain as string type) - as it used to be in the original connector.

dviry avatar Dec 15 '23 20:12 dviry

This issue is stale because it has been open for 90 days with no activity.

github-actions[bot] avatar Jun 08 '24 01:06 github-actions[bot]

Would it be helpful to add a [BsonRepresentation(BsonType.ObjectId)] attribute to the Id property? Only asking this because when I attempt a semantic search against MongoDB, I am getting An error occurred while deserializing the Id property of class Microsoft.SemanticKernel.Connectors.MongoDB.MongoDBMemoryEntry: Cannot deserialize a 'String' from BsonType 'ObjectId'.

williambohrmann3 avatar Sep 30 '24 22:09 williambohrmann3

Closing as we have new memory abstractions and will be obsoleting the old ones in the future.

markwallace-microsoft avatar Nov 25 '24 12:11 markwallace-microsoft