The embedding remains stored even when I delete the IDs of those documents

[tokiou]

What happened?

When I attempt deletion using this method:

docs = collection.get(where={"source": 'www.something.com'})
ids = docs.get('ids')
for id in ids:
    collection.delete(id)

It appears that the deletion is successful. However, for some reason, the deleted data seems to persist in the SQLite Chroma database. Consequently, I'm unable to index new data. Whenever I attempt to index, I encounter a warning, and no embeddings are being stored: WARNING:chromadb.segment.impl.vector.local_persistent_hnsw:Add of existing embedding ID:

Versions

Chroma version : 0.4.24

Relevant log output

No response

[tazarov]

@tokiou, we have a small bug that ends up displaying these messages - Here's a PR that attempts to solve the issue - https://github.com/chroma-core/chroma/pull/1763

I think it is safe to ignore these messages for now.

[tazarov]

@tokiou, found the root cause for this, #1763 is most definitely not the fix.