pymilvus icon indicating copy to clipboard operation
pymilvus copied to clipboard

Creating an index with invalid index_type throws an TypeError exception

Open ThreadDao opened this issue 4 years ago • 16 comments

Describe the bug A clear and concise description of what the bug is. Creating index with index_params which index_type is list, dict or int (not string) will encounter TypeError

Steps/Code to reproduce behavior Follow this guide to craft a minimal bug report. This helps us reproduce the issue you're having and resolve the issue more quickly.

@pytest.mark.parametrize("index_type", ct.get_invalid_strs)
    def test_index_type_invalid(self, index_type):
        c_name = cf.gen_unique_str(prefix)
        collection_w = self.init_collection_wrap(name=c_name)
        index_params = copy.deepcopy(default_index_params)
        index_params["index_type"] = index_type
        if not isinstance(index_params["index_type"], str):
            msg = "must be str"
        else:
            msg = "Invalid index_type"
        self.index_wrap.init_index(collection_w.collection, default_field_name, index_params,
                                   check_task=CheckTasks.err_res,
                                   check_items={ct.err_code: 1, ct.err_msg: msg})
File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/stub.py", line 738, in create_index
    raise ParamError("Invalid index_type: " + params['index_type'] +
TypeError: can only concatenate str (not "list") to str  (api_request.py:26)

TypeError: can only concatenate str (not "int") to str
TypeError: can only concatenate str (not "tuple") to str
TypeError: can only concatenate str (not "dict") to str
TypeError: can only concatenate str (not "NoneType") to str

Expected behavior A clear and concise description of what you expected to happen. Throws an exception with error code and message

Environment details

  • Hardware/Softward conditions (OS, CPU, GPU, Memory)
  • Method of installation (Docker, or from source)
  • Milvus version (v0.3.1, or v0.4.0) pymilvus 2.0.0rc5.dev29
  • Milvus configuration (Settings you made in server_config.yaml)

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

ThreadDao avatar Aug 31 '21 02:08 ThreadDao

This is because params['index_type'] is not string type, and '+' can only be used between string and string. There is many problems like this in project.

xiaocai2333 avatar Sep 16 '21 09:09 xiaocai2333

/good-first-issue

xiaocai2333 avatar Sep 16 '21 09:09 xiaocai2333

@xiaocai2333: This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-good-first-issue command.

In response to this:

/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

sre-ci-robot avatar Sep 16 '21 09:09 sre-ci-robot

This issue addresses drawbacks on PyMilvus exception handling.

Working on it now

XuanYang-cn avatar Sep 18 '21 06:09 XuanYang-cn

This case has the same problem

@pytest.mark.xfail(reason="pymilvus issue #677")
    def test_loading_progress_invalid_partition_names(self, get_invalid_partition_names):
        """
        target: test loading progress with invalid partition names
        method: input invalid partition names
        expected: raise an exception
        """
        collection_w = self.init_collection_general(prefix)[0]
        partition_names = get_invalid_partition_names
        err_msg = {ct.err_code: 0, ct.err_msg: "`partition_name_array` value {} is illegal".format(partition_names)}
        collection_w.load()
        self.utility_wrap.loading_progress(collection_w.name, partition_names,
                                           check_task=CheckTasks.err_res, check_items=err_msg)

Actual results (python 3.8):

2021-09-22 11:54:47,528 - DEBUG - ci_test]: (api_request)  : [loading_progress] args: ['utility_k8DQyxcK', (1,), 'default'], kwargs: {} (api_request.py:44)
[2021-09-22 11:54:47,542 - ERROR - ci_test]: Traceback (most recent call last):
  File "/home/zong/zong/projects/milvus/tests/python_client/utils/api_request.py", line 18, in inner_wrapper
    res = func(*args, **kwargs)
  File "/home/zong/zong/projects/milvus/tests/python_client/utils/api_request.py", line 45, in api_request
    return func(*arg, **kwargs)
  File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/orm/utility.py", line 61, in loading_progress
    return _get_connection(using).load_partitions_progress(collection_name, partition_names)
  File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/stub.py", line 60, in handler
    raise e
  File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/stub.py", line 44, in handler
    return func(self, *args, **kwargs)
  File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/stub.py", line 1220, in load_partitions_progress
    return handler.load_partitions_progress(collection_name, partition_names, timeout=timeout)
  File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 82, in handler
    raise e
  File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 59, in handler
    return func(self, *args, **kwargs)
  File "/home/zong/zong/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 1033, in load_partitions_progress
    msg = "partitionID of partitionName:" + name + " can not be found"
TypeError: can only concatenate str (not "int") to str

ThreadDao avatar Sep 22 '21 03:09 ThreadDao

/unassign /assign @ThreadDao Please help verify in pymilvus 2.1.0.dev90

XuanYang-cn avatar Jun 27 '22 10:06 XuanYang-cn

@XuanYang-cn

Reproduced in pymilvus: 2.2.0.dev6

[2022-08-09 09:49:37 - DEBUG - ci_test]: (api_request)  : [Index] args: [<Collection>:
-------------
<name>: index_vm9UHj10
<partitions>: [{"name": "_default", "collection_name": "index_vm9UHj10", "description": ""}]
<description>:
<schema>: {
  auto_id: False
  description:
  fields: [{
    name: int64
    description:
    type: 5
    is_primary: True
    auto_id: F......, kwargs: {'timeout': 40, 'index_name': '_default_idx'} (api_request.py:56)
[2022-08-09 09:49:37 - ERROR - pymilvus.decorators]: Unexcepted error: [create_index], can only concatenate str (not "NoneType") to str, <Time: {'RPC start': '2022-08-09 09:49:37.151498', 'Exception': '2022-08-09 09:49:37.152666'}> (decorators.py:107)
[2022-08-09 09:49:37 - ERROR - ci_test]: Traceback (most recent call last):
  File "/root/binbin/python_client/utils/api_request.py", line 26, in inner_wrapper
    res = func(*args, **_kwargs)
  File "/root/binbin/python_client/utils/api_request.py", line 57, in api_request
    return func(*arg, **kwargs)
  File "/root/virtualenv/milvus-binbin-1/lib/python3.8/site-packages/pymilvus/orm/index.py", line 80, in __init__
    conn.create_index(self._collection.name, self._field_name, self._index_params, **self._kwargs)
  File "/root/virtualenv/milvus-binbin-1/lib/python3.8/site-packages/pymilvus/decorators.py", line 108, in handler
    raise e
  File "/root/virtualenv/milvus-binbin-1/lib/python3.8/site-packages/pymilvus/decorators.py", line 92, in handler
    return func(*args, **kwargs)
  File "/root/virtualenv/milvus-binbin-1/lib/python3.8/site-packages/pymilvus/decorators.py", line 74, in handler
    raise e
  File "/root/virtualenv/milvus-binbin-1/lib/python3.8/site-packages/pymilvus/decorators.py", line 48, in handler
    return func(self, *args, **kwargs)
  File "/root/virtualenv/milvus-binbin-1/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 540, in create_index
    check_index_params(params)
  File "/root/virtualenv/milvus-binbin-1/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 511, in check_index_params
    raise ParamError("Invalid index_type: " + params['index_type'] +
TypeError: can only concatenate str (not "NoneType") to str
 (api_request.py:39)
[2022-08-09 09:49:37 - ERROR - ci_test]: (api_response) : can only concatenate str (not "NoneType") to str (api_request.py:40)

binbinlv avatar Aug 09 '22 09:08 binbinlv

/assign @XuanYang-cn

binbinlv avatar Aug 09 '22 09:08 binbinlv

/unassign @ThreadDao

binbinlv avatar Aug 09 '22 09:08 binbinlv

/unassign /assign @binbinlv

fixed @ pymilvus2.2.0.dev29

XuanYang-cn avatar Sep 08 '22 08:09 XuanYang-cn

OK, verifying。

binbinlv avatar Sep 08 '22 08:09 binbinlv

@NicoYuan1986 Could you please verify this issue? Thanks.

binbinlv avatar Sep 08 '22 08:09 binbinlv

@NicoYuan1986 Could you please verify this issue? Thanks.

Get.

NicoYuan1986 avatar Sep 08 '22 08:09 NicoYuan1986

The issue has been solved.

NicoYuan1986 avatar Sep 13 '22 06:09 NicoYuan1986

@ThreadDao This issue has been fixed, could you please close this issue? Thanks.

binbinlv avatar Sep 13 '22 06:09 binbinlv

/assign @ThreadDao /unassign

binbinlv avatar Sep 13 '22 06:09 binbinlv