milvus icon indicating copy to clipboard operation
milvus copied to clipboard

[Bug]: Collection release gets error CollectionNotExists after granting release privilege with db

Open ThreadDao opened this issue 1 year ago • 5 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Environment

- Milvus version: PR-23742-20230505-3a859f674
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka): rocksmq   
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 2.2.9.dev3
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

  1. root user connect
  2. create a user -> create a role -> create a db
  3. create a collection utility_JBC2KL1p
  4. grant privilege (Collection, utility_JBC2KL1p, Release) to the role with the db
  5. collection insert -> index -> load
  6. disconnect -> connect with granted user, specify db
  7. release collection gets exception
pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=code: UnexpectedError, reason: code: CollectionNotExists, reason: can't find collection: utility_JBC2KL1p)>

Expected Behavior

release success

Steps To Reproduce

@pytest.mark.tags(CaseLabel.L3)
    @pytest.mark.parametrize("with_db", [False, True])
    def test_verify_grant_collection_release_privilege(self, host, port, with_db):
        """
        target: verify grant collection release privilege
        method: verify grant collection release privilege
        expected: verify success
        """
        self.connection_wrap.connect(host=host, port=port, user=ct.default_user,
                                     password=ct.default_password, check_task=ct.CheckTasks.ccr)
        user = cf.gen_unique_str(prefix)
        password = cf.gen_unique_str(prefix)
        c_name = cf.gen_unique_str(prefix)
        r_name = cf.gen_unique_str(prefix)
        self.utility_wrap.init_role(r_name)
        self.utility_wrap.create_role()
        u, _ = self.utility_wrap.create_user(user=user, password=password)
        self.utility_wrap.role_add_user(user)

        db_kwargs = {}
        if with_db:
            db_name = cf.gen_unique_str("db")
            self.database_wrap.create_database(db_name)
            db_kwargs = {"db_name": db_name}

        self.utility_wrap.role_grant("Collection", c_name, "Release", **db_kwargs)
        collection_w = self.init_collection_wrap(name=c_name)
        data = cf.gen_default_list_data(100)
        mutation_res, _ = collection_w.insert(data=data)
        collection_w.create_index(ct.default_float_vec_field_name, ct.default_flat_index)
        collection_w.load()
        self.utility_wrap.role_list_grants(**db_kwargs)
        self.connection_wrap.disconnect(alias=DefaultConfig.DEFAULT_USING)
        self.connection_wrap.connect(host=host, port=port, user=user,
                                     password=password, check_task=ct.CheckTasks.ccr, **db_kwargs)

        collection_w.release()


### Milvus Log

_No response_

### Anything else?

_No response_

ThreadDao avatar May 08 '23 08:05 ThreadDao

/assign @jaime0815 /unassign

yanliang567 avatar May 08 '23 11:05 yanliang567

The same problem also exists on other privilege of the Collection object. After you fixed, please filter the case in test_utility.py by @pytest.mark.skip(reason="https://github.com/milvus-io/milvus/issues/23943" and run the case. tnx

ThreadDao avatar May 09 '23 03:05 ThreadDao

/assign @SimFG /unassign

jaime0815 avatar May 09 '23 03:05 jaime0815

@ThreadDao it has been fixed by @jaime0815, you can use the latest image to checkout it. In addition, this problem is not related to the rbac function

SimFG avatar May 09 '23 03:05 SimFG

/unassign

SimFG avatar May 12 '23 06:05 SimFG

/assign @jaime0815 Not fix yet

ThreadDao avatar May 17 '23 13:05 ThreadDao

Please use the latest 2.2 image to verify @ThreadDao /unassign

jaime0815 avatar May 22 '23 09:05 jaime0815

@ThreadDao When you create the collection, you should give a db param in the CreateCollectionRequest. If you not, the collection will be related with the default db.

In this case, when creating the collection, the db param isn't specified, which means the collection is related with the default db. And then, after connecting the random db, you can't release the collection, because this collection does not belong to the random db, but the default db.

cc @jaime0815

SimFG avatar May 22 '23 09:05 SimFG

@jaime0815 If you fix the issue, you can run all the marked @pytest.mark.skip(reason="https://github.com/milvus-io/milvus/issues/23943") case to verify this

  1. role list grant
[2023-05-22 22:54:21,791 - DEBUG - ci_test]: (api_request)  : [Role.list_grants] args: ['db_dAVOxaQg'], kwargs: {} (api_request.py:56)
[2023-05-22 22:54:21,813 - DEBUG - ci_test]: (api_response) : GrantInfo groups:
- GrantItem: <object:Collection>, <object_name:utility_9fgvZKjn>, <db_name:db_dAVOxaQg>, <role_name:utility_1H0GK8Y2>, <grantor_name:root>, <privilege:Insert>  (api_request.py:31)
  1. user connect
[2023-05-22 22:54:21,814 - DEBUG - ci_test]: (api_request)  : [Connections.disconnect] args: ['default'], kwargs: {} (api_request.py:56)
[2023-05-22 22:54:21,815 - DEBUG - ci_test]: (api_response) : None  (api_request.py:31)
[2023-05-22 22:54:21,816 - DEBUG - ci_test]: (api_request)  : [Connections.connect] args: ['default', 'utility_aLUNwQDV', 'utility_7P5iNjv8', 'db_dAVOxaQg'], kwargs: {'host': '10.101.63.90', 'port': 19530} (api_request.py:56)
[2023-05-22 22:54:21,972 - DEBUG - ci_test]: (api_response) : None  (api_request.py:31)
[2023-05-22 22:54:22,332 - DEBUG - ci_test]: (api_request)  : [Collection.insert] args: [[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,......, kwargs: {'timeout': 120} (api_request.py:56)
[2023-05-22 22:54:24,791 - ERROR - ci_test]: Traceback (most recent call last):
  File "/Users/nausicca/projects/milvus/tests/python_client/utils/api_request.py", line 26, in inner_wrapper
    res = func(*args, **_kwargs)
  File "/Users/nausicca/projects/milvus/tests/python_client/utils/api_request.py", line 57, in api_request
    return func(*arg, **kwargs)
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/orm/collection.py", line 436, in insert
    res = conn.batch_insert(self._name, entities, partition_name,
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/decorators.py", line 109, in handler
    raise e
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/decorators.py", line 105, in handler
    return func(*args, **kwargs)
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/decorators.py", line 136, in handler
    ret = func(self, *args, **kwargs)
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/decorators.py", line 85, in handler
    raise e
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/decorators.py", line 50, in handler
    return func(self, *args, **kwargs)
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 415, in batch_insert
    raise err
  File "/Users/nausicca/.virtualenvs/milvus/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 411, in batch_insert
    raise MilvusException(response.status.error_code, response.status.reason)
pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=code: UnexpectedError, reason: code: CollectionNotExists, reason: can't find collection: utility_9fgvZKjn)>
 (api_request.py:39)
[2023-05-22 22:54:24,792 - ERROR - ci_test]: (api_response) : <MilvusException: (code=1, message=code: UnexpectedError, reason: code: CollectionNotExists, reason: can't find collection: utility_9fgvZKjn)> (api_request.py:40)
FAILED

ThreadDao avatar May 22 '23 14:05 ThreadDao

/assign @ThreadDao

longjiquan avatar May 23 '23 08:05 longjiquan

Fixed 2.2.0-20230522-9569e7ee

ThreadDao avatar May 23 '23 12:05 ThreadDao