milvus icon indicating copy to clipboard operation
milvus copied to clipboard

[Bug]: [Nightly]Lack error detection of invalid auto_id

Open NicoYuan1986 opened this issue 1 year ago • 2 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Environment

- Milvus version: d163b77
- Deployment mode(standalone or cluster): cluster
- MQ type(rocksmq, pulsar or kafka):    kafka
- SDK version(e.g. pymilvus v2.0.0rc2): 2.2.9.dev28
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Lack error detection of invalid auto_id.

[2023-05-31T21:20:22.187Z] ________ TestCollectionParams.test_collection_invalid_auto_id[auto_id3] ________
[2023-05-31T21:20:22.187Z] [gw1] linux -- Python 3.8.16 /usr/local/bin/python3
[2023-05-31T21:20:22.187Z] 
[2023-05-31T21:20:22.187Z] self = <test_collection.TestCollectionParams object at 0x7f1bf7118bb0>
[2023-05-31T21:20:22.187Z] auto_id = (1,)
[2023-05-31T21:20:22.187Z] 
[2023-05-31T21:20:22.187Z]     @pytest.mark.tags(CaseLabel.L2)
[2023-05-31T21:20:22.187Z]     @pytest.mark.parametrize("auto_id", ct.get_invalid_strs)
[2023-05-31T21:20:22.187Z]     def test_collection_invalid_auto_id(self, auto_id):
[2023-05-31T21:20:22.187Z]         """
[2023-05-31T21:20:22.187Z]         target: test collection with invalid auto_id
[2023-05-31T21:20:22.187Z]         method: define field with auto_id=non-bool
[2023-05-31T21:20:22.187Z]         expected: raise exception
[2023-05-31T21:20:22.187Z]         """
[2023-05-31T21:20:22.187Z]         self._connect()
[2023-05-31T21:20:22.187Z]         int_field = cf.gen_int64_field(is_primary=True)
[2023-05-31T21:20:22.187Z]         vec_field = cf.gen_float_vec_field(name='vec')
[2023-05-31T21:20:22.187Z]         error = {ct.err_code: 0, ct.err_msg: "Param auto_id must be bool type"}
[2023-05-31T21:20:22.187Z] >       self.collection_schema_wrap.init_collection_schema([int_field, vec_field], auto_id=auto_id,
[2023-05-31T21:20:22.187Z]                                                            check_task=CheckTasks.err_res, check_items=error)
[2023-05-31T21:20:22.187Z] 
[2023-05-31T21:20:22.187Z] testcases/test_collection.py:777: 
[2023-05-31T21:20:22.187Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2023-05-31T21:20:22.187Z] base/schema_wrapper.py:17: in init_collection_schema
[2023-05-31T21:20:22.187Z]     check_result = ResponseChecker(response, func_name, check_task, check_items, is_succ=is_succ, fields=fields,
[2023-05-31T21:20:22.187Z] check/func_check.py:38: in run
[2023-05-31T21:20:22.187Z]     result = self.assert_exception(self.response, self.succ, self.check_items)
[2023-05-31T21:20:22.187Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2023-05-31T21:20:22.187Z] 
[2023-05-31T21:20:22.187Z] res = {'auto_id': (1,), 'description': '', 'fields': [{'name': 'int64', 'description': '', 'type': <DataType.INT64: 5>, 'is_...e, 'auto_id': (1,)}, {'name': 'vec', 'description': '', 'type': <DataType.FLOAT_VECTOR: 101>, 'params': {'dim': 128}}]}
[2023-05-31T21:20:22.187Z] actual = True
[2023-05-31T21:20:22.187Z] error_dict = {'err_code': 0, 'err_msg': 'Param auto_id must be bool type'}
[2023-05-31T21:20:22.187Z] 
[2023-05-31T21:20:22.187Z]     @staticmethod
[2023-05-31T21:20:22.187Z]     def assert_exception(res, actual=True, error_dict=None):
[2023-05-31T21:20:22.187Z] >       assert actual is False
[2023-05-31T21:20:22.187Z] E       AssertionError
[2023-05-31T21:20:22.187Z] 
[2023-05-31T21:20:22.187Z] check/func_check.py:101: AssertionError
[2023-05-31T21:20:22.187Z] ------------------------------ Captured log setup ------------------------------
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - INFO - ci_test]: *********************************** setup *********************************** (client_base.py:42)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - INFO - ci_test]: [setup_method] Start setup test case test_collection_invalid_auto_id. (client_base.py:43)
[2023-05-31T21:20:22.187Z] ------------------------------ Captured log call -------------------------------
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_request)  : [Connections.connect] args: ['default', '', '', ''], kwargs: {'host': 'mdk-207-n-milvus.milvus-ci', 'port': '19530'} (api_request.py:56)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_response) : None  (api_request.py:31)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_request)  : [FieldSchema] args: ['int64', <DataType.INT64: 5>, ''], kwargs: {'is_primary': True} (api_request.py:56)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_response) : {'name': 'int64', 'description': '', 'type': <DataType.INT64: 5>, 'is_primary': True, 'auto_id': False}  (api_request.py:31)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_request)  : [FieldSchema] args: ['vec', <DataType.FLOAT_VECTOR: 101>, ''], kwargs: {'dim': 128, 'is_primary': False} (api_request.py:56)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_response) : {'name': 'vec', 'description': '', 'type': <DataType.FLOAT_VECTOR: 101>, 'params': {'dim': 128}}  (api_request.py:31)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_request)  : [CollectionSchema] args: [[{'name': 'int64', 'description': '', 'type': <DataType.INT64: 5>, 'is_primary': True, 'auto_id': False}, {'name': 'vec', 'description': '', 'type': <DataType.FLOAT_VECTOR: 101>, 'params': {'dim': 128}}], ''], kwargs: {'auto_id': (1,)} (api_request.py:56)
[2023-05-31T21:20:22.187Z] [2023-05-31 20:04:34 - DEBUG - ci_test]: (api_response) : {'auto_id': (1,), 'description': '', 'fields': [{'name': 'int64', 'description': '', 'type': <DataType.INT64: 5>, 'is_primary': True, 'auto_id': (1,)}, {'name': 'vec', 'description': '', 'type': <DataType.FLOAT_VECTOR: 101>, 'params': {'dim': 128}}]}  (api_request.py:31)

Expected Behavior

raise exception

Steps To Reproduce

No response

Milvus Log

  1. link: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI/detail/2.2.0/207/pipeline/140/
  2. log: artifacts-milvus-distributed-kafka-nightly-207-pymilvus-e2e-logs.tar.gz

Anything else?

No response

NicoYuan1986 avatar Jun 01 '23 02:06 NicoYuan1986

/assign @XuanYang-cn

binbinlv avatar Jun 01 '23 02:06 binbinlv

Another case: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI/detail/2.2.0/208/pipeline/156/

  1. log: artifacts-milvus-distributed-kafka-nightly-208-pymilvus-e2e-logs.tar.gz
  2. failed time: [2023-06-01T20:04:21.355Z] [gw0] [ 3%] FAILED testcases/test_collection.py::TestCollectionParams::test_collection_auto_id_inconsistent

NicoYuan1986 avatar Jun 02 '23 02:06 NicoYuan1986

Fixed.

NicoYuan1986 avatar Jun 14 '23 03:06 NicoYuan1986