qdrant-client
qdrant-client copied to clipboard
How to pytest for collection config with QdrantLocal?
Hi, we want to test in CI/CD, that a collection does have expected configuration values.
from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance,
client = QdrantClient(':memory:') # same with client = QdrantClient(path="qdrant_data")
client.create_collection(
collection_name="my_collection",
on_disk_payload=False,
vectors_config=VectorParams(size=100, distance=Distance.COSINE)
)
collection = client.get_collection("my_collection")
assert collection.config.params.on_disk_payload == False # Fails
assert collection.config.params.on_disk_payload == None # Succeeds
I want the first assertion to succeed. Is this somehow possible? for QdrantClient(path="qdrant_data") there is a meta.json written to the qdrant_data folder. Couldn't there end up also the passed configuration?
Our use case:
We create a collection that is loaded into memory. When we create a new collection, we want to keep the old collection and label it as backup collection. The backup collection should be moved to disk.
Now we want to test with pytest that the config changes and e. g. on_disk_payload is set to True after it is labeled as backup collection.
cp from discord Local mode ignores all additional parameters like hnsw config, optimizers config, etc. however it saves those parameters which were provided in vectors_config e.g. input:
cl.create_collection('col', vectors_config=models.VectorParams(size=10, distance=models.Distance.COSINE, hnsw_config=models.HnswConfigDiff(m=32, ef_construct=160)))
output:
CollectionInfo(status=<CollectionStatus.GREEN: 'green'>, optimizer_status=<OptimizersStatusOneOf.OK: 'ok'>, vectors_count=0, indexed_vectors_count=0, points_count=0, segments_count=1, config=CollectionConfig(params=CollectionParams(vectors=VectorParams(size=10, distance=<Distance.COSINE: 'Cosine'>, hnsw_config=HnswConfigDiff(m=32, ef_construct=160, full_scan_threshold=None, max_indexing_threads=None, on_disk=None, payload_m=None), quantization_config=None, on_disk=None), shard_number=None, replication_factor=None, write_consistency_factor=None, on_disk_payload=None), hnsw_config=HnswConfig(m=16, ef_construct=100, full_scan_threshold=10000, max_indexing_threads=0, on_disk=None, payload_m=None), optimizer_config=OptimizersConfig(deleted_threshold=0.2, vacuum_min_vector_number=1000, default_segment_number=0, max_segment_size=None, memmap_threshold=None, indexing_threshold=20000, flush_interval_sec=5, max_optimization_threads=1), wal_config=WalConfig(wal_capacity_mb=32, wal_segments_ahead=0), quantization_config=None), payload_schema={})