chroma icon indicating copy to clipboard operation
chroma copied to clipboard

[Bug]: Cannot update metadata value to None

Open Russell-Pollari opened this issue 2 years ago • 2 comments

What happened?

client = chromadb.Client()
collection = client.create_collection("test")

collection.add(
    embeddings=[[1.0, 1.1]],
    ids=["id_1"],
    metadatas=[{"string_value": "please_remove_me"}],
)

collection.update(ids=["id_1"], metadatas=[{"string_value": None}])

# Value Error:  Expected metadata value to be a str, int, or float, or list got None which is a <class 'NoneType'>

Looking at other parts of the codebase, I would expect updating metadata value to None to remove it

e.g. the types and validators imply None should be allowed for metadata updates UpdateMetadata = Mapping[str, Union[int, float, str, None]]

And the SQLite metadata segment has the correct type and logic:

    def _update_metadata(self, cur: Cursor, id: int, metadata: UpdateMetadata) -> None:
        """Update the metadata for a single EmbeddingRecord"""
        t = Table("embedding_metadata")
        to_delete = [k for k, v in metadata.items() if v is None]
        if to_delete:

In short: Collection.update should use UpdateMetadata and validate_update_metadata

Versions

Chroma v0.4.0 Python 3.8.10

Relevant log output

Traceback (most recent call last):
  File "testing.py", line 13, in <module>
    collection.update(ids=["id_1"], metadatas=[{"string_value": None}])
  File "/home/russell/projects/chroma/chromadb/api/models/Collection.py", line 272, in update
    ids, embeddings, metadatas, documents = self._validate_embedding_set(
  File "/home/russell/projects/chroma/chromadb/api/models/Collection.py", line 357, in _validate_embedding_set
    validate_metadatas(maybe_cast_one_to_many(metadatas))
  File "/home/russell/projects/chroma/chromadb/api/types.py", line 181, in validate_metadatas
    validate_metadata(metadata)
  File "/home/russell/projects/chroma/chromadb/api/types.py", line 140, in validate_metadata
    raise ValueError(
ValueError: Expected metadata value to be a str, int, or float, or list got None which is a <class 'NoneType'>

Russell-Pollari avatar Jul 18 '23 14:07 Russell-Pollari