pymilvus icon indicating copy to clipboard operation
pymilvus copied to clipboard

[Bug]: Upsert on collection with Dynamic Schema enabled

Open vrtornisiello opened this issue 1 year ago • 3 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Describe the bug

When I try to upsert an entity on a collection that has dynamic schema enabled, I see the error message attached below:

error message

Expected Behavior

When I do an upsert on a collection that has dynamic schema enabled, it should succeed.

Steps/Code To Reproduce behavior

from pymilvus import connections, Collection, FieldSchema, CollectionSchema, DataType, utility

connections.connect(host='localhost', port='19530')

COL_NAME = 'test_upsert_on_dynamic'

if utility.has_collection(COL_NAME):
    collection = Collection(COL_NAME)
else:
    # 1. define fields
    fields = [
        FieldSchema(name="id", dtype=DataType.VARCHAR, max_length=36, is_primary=True),
        FieldSchema(name="title", dtype=DataType.VARCHAR, max_length=512),
        FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=512)
    ]

    # 2. enable dynamic schema in schema definition
    schema = CollectionSchema(
        fields,
        'Testing upsert on Dynamic Schema',
        enable_dynamic_field=True
    )

    # 3. reference the schema in a collection
    collection = Collection(COL_NAME, schema)

    # 4. index the vector field and load the collection
    index_params = {
        'index_type': 'AUTOINDEX',
        'metric_type': 'L2',
        'params': {}
    }

    collection.create_index(
        field_name='embedding',
        index_params=index_params
    )

# 5. load the collection
collection.load()

# Insert some data
import random
import uuid

entities = [
    {
        'id': str(uuid.uuid4()),
        'title': f'title_{i}',
        'embedding': [random.random() for _ in range(512)]
    } for i in range(10)
]

collection.insert(entities)
collection.flush()

# Query the data which will be upserted
query_results = collection.query(
    expr=f"title == 'title_0'",
    output_fields=['title', 'embedding']
)

# Modify data
for res in query_results:
    res['title'] = 'Modified Title'

# Upsert
collection.upsert(query_results)

Environment details

- Ubuntu 20.04 LTS
- Milvus version: 2.3.0 (Standalone)
- Installation method: Docker (CPU version)
- PyMilvus version: 2.3.1

Anything else?

No response

vrtornisiello avatar Oct 05 '23 18:10 vrtornisiello

/assign @smellthemoon

XuanYang-cn avatar Oct 08 '23 03:10 XuanYang-cn

/assign

smellthemoon avatar Oct 08 '23 03:10 smellthemoon

The issue related with #26595, you can use the milvus 2.3.1 version and try it again. Thanks! @vrtornisiello

smellthemoon avatar Oct 08 '23 03:10 smellthemoon