[Bug]: System cache not cleared when a new database is loaded with same persitent_directory

[sjoerd222888]

What happened?

I create a database locally and store it on the filesystem. I am using persistent_directory to define the storage location. I have multiple database like this, I usually store them as a tar.gz archive. Now I restore a certain database (=unpack the tar.gz) and load the database.

What happens is the following: The first database I unpack works fine. But then I unpack a second archive and create a new instance on chroma then under the hood the cache still contains data from the old database resulting sometimes in exception like:

ValidationError(model='Document', errors=[{'loc': ('page_content',), 'msg': 'none is not an allowed value', 'type': 'type_error.none.not_allowed'}]) 

or

InvalidCollectionException('Collection d45cbd56-f13a-4c67-8237-f0769592eb2c does not exist.')

Which seem linked to the fact that the client system cache contains still old data. To work around it I can is calling clear_system_cache() on the client. However as you can see here the method is annotated with documentation stating it should only be used for testing purpose.

I would say the actual bug is that old data remains in the cache that should not.

Calling clear_system_cache() is valid workaround but it would be nice to officially document it as a full API functionality and not a method only intended for testing purpose.

Versions

Chroma v0.5.3, Python 3.11.8, MacOS 14.5

Relevant log output

No response