[Bug]: [Dynamic] "is_dynamic" in field schema could be set, and displayed in collection properties

[binbinlv]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version: master/2.2 latest
- Deployment mode(standalone or cluster): both
- MQ type(rocksmq, pulsar or kafka):    all
- SDK version(e.g. pymilvus v2.0.0rc2): master/2.2 latest
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

"is_dynamic" in field schema could be set, and displayed in collection properties

>>> int64_field = FieldSchema(name="int64", dtype=DataType.INT64, is_primary=True, is_dynamic=True)
>>> float_vector = FieldSchema(name="float_vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
>>> schema = CollectionSchema(fields=[int64_field, float_vector], enable_dynamic_field=True)
>>> collection.describe()
{'collection_name': 'test_search_collection_binbin_tmp_0', 'auto_id': False, 'num_shards': 1, 'description': '', 'fields': [{'field_id': 100, 'name': 'int64', 'description': '', 'type': 5, 'params': {}, 'is_primary': True, 'auto_id': False, 'is_partition_key': False, 'default_value': , 'is_dynamic': False}, {'field_id': 101, 'name': 'float_vector', 'description': '', 'type': 101, 'params': {'dim': 128}, 'is_primary': False, 'auto_id': False, 'is_partition_key': False, 'default_value': , 'is_dynamic': False}], 'aliases': [], 'collection_id': 441826783185452180, 'consistency_level': 2, 'properties': [], 'num_partitions': 0, 'enable_dynamic_field': True}

Expected Behavior

I think "is_dynamic" is for internal implementation usage, not for users. If expose this property, it may confuse users. And actually this field could not be set by users, it is always false no matter what is set.

So may we hide this field and not expose this to users?

Steps To Reproduce

from pymilvus import CollectionSchema, FieldSchema
from pymilvus import Collection
from pymilvus import connections
from pymilvus import DataType
from pymilvus import Partition
from pymilvus import utility

connections.connect()

int64_field = FieldSchema(name="int64", dtype=DataType.INT64, is_primary=True, is_dynamic=True)
dim = 128
float_vector = FieldSchema(name="float_vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
schema = CollectionSchema(fields=[int64_field, float_vector], enable_dynamic_field=True)
collection = Collection("test_search_collection_binbin_tmp_0", schema=schema)
collection.describe()

Milvus Log

No response

Anything else?

No response

[xiaocai2333]

/assign @binbinlv fixed it, please verify

[binbinlv]

Now is_dynamic could not be set: pymilvus: 2.4.0.dev67 milvus: master-20230602-6b1712e0

>>> int64_field = FieldSchema(name="int64", dtype=DataType.INT64, is_primary=True, is_dynamic=True)
>>> dim = 128
>>> float_vector = FieldSchema(name="float_vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
>>> schema = CollectionSchema(fields=[int64_field, float_vector], enable_dynamic_field=True)
>>> collection = Collection("test_search_collection_binbin_tmp_0", schema=schema)
RPC error: [create_collection], <MilvusException: (code=1, message=cannot explicitly set a field as a dynamic field)>, <Time:{'RPC start': '2023-06-02 17:10:04.157019', 'RPC error': '2023-06-02 17:10:04.251477'}>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/orm/collection.py", line 123, in __init__
    conn.create_collection(self._name, schema, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 109, in handler
    raise e
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 105, in handler
    return func(*args, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 136, in handler
    ret = func(self, *args, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 85, in handler
    raise e
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 50, in handler
    return func(self, *args, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 252, in create_collection
    raise MilvusException(status.error_code, status.reason)
pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=cannot explicitly set a field as a dynamic field)>

pymilvus:2.2.9.dev36 milvus:2.2.0-20230602-9ffcd53b

>>> int64_field = FieldSchema(name="int64", dtype=DataType.INT64, is_primary=True, is_dynamic=True)
>>> dim = 128
>>> float_vector = FieldSchema(name="float_vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
>>> schema = CollectionSchema(fields=[int64_field, float_vector], enable_dynamic_field=True)
>>> collection = Collection("test_search_collection_binbin_tmp_0", schema=schema)
RPC error: [create_collection], <MilvusException: (code=1, message=cannot explicitly set a field as a dynamic field)>, <Time:{'RPC start': '2023-06-02 17:15:17.342529', 'RPC error': '2023-06-02 17:15:17.418145'}>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/orm/collection.py", line 123, in __init__
    conn.create_collection(self._name, schema, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 109, in handler
    raise e
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 105, in handler
    return func(*args, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 136, in handler
    ret = func(self, *args, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 85, in handler
    raise e
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/decorators.py", line 50, in handler
    return func(self, *args, **kwargs)
  File "/Users/binbin/towhee-test-env-3.8-0.9.0/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 251, in create_collection
    raise MilvusException(status.error_code, status.reason)
pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=cannot explicitly set a field as a dynamic field)>

[binbinlv]

@xiaocai2333 "is_dynamic" still be kept, is it necessary to keep this property?

[yanliang567]

I think the doc shall be updated: there is no is_dynamic parameter in field schema. https://milvus.io/docs/schema.md @xiaocai2333 could you please double confirm that? @AnthonyTsu1984 could you please update the doc after @xiaocai2333 confirmed. /assign @AnthonyTsu1984

[xiaocai2333]

We currently do not support specifying 'is_dynamic' on the schema column. is_dynamic will not be displayed.

[xiaocai2333]

@binbinlv

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[xiaocai2333]

@binbinlv please verify it and close it.

[binbinlv]

Now "is_dynamic" is not displayed in the result of collection.describe(), verified and fixed.

>>> collection.describe()
{'collection_name': 'test_search_collection_binbin_tmp_0', 'auto_id': False, 'num_shards': 1, 'description': '', 'fields': [{'field_id': 100, 'name': 'int64', 'description': '', 'type': 5, 'params': {}, 'is_primary': True}, {'field_id': 101, 'name': 'float_vector', 'description': '', 'type': 101, 'params': {'dim': 128}}], 'aliases': [], 'collection_id': 443591954959714527, 'consistency_level': 2, 'properties': [], 'num_partitions': 1, 'enable_dynamic_field': True}