milvus
milvus copied to clipboard
milvus-io
[Bug]: Attempting to release an unloaded partition causes the collection can't release any more
Is there an existing issue for this?
- [X] I have searched the existing issues
Environment
- Milvus version: 2.2.5
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka): default
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 2.2.4
- OS(Ubuntu or CentOS): CentOS
- CPU/Memory: 2 CPU, 4G Memory
- GPU: None
- Others:
Current Behavior
When I try to release an unloaded partition using python code, the program is blocked. I cannot release the collection any more unless I restart Milvus.
Expected Behavior
The program could throw an exception when I try to release an unloaded partition and don't cause the collection in a bad state.
Steps To Reproduce
1. Install and Run Milvus Standalone with Docker Compose using following command:
$ wget https://github.com/milvus-io/milvus/releases/download/v2.2.5/milvus-standalone-docker-compose.yml -O docker-compose.yml
sudo docker-compose up -d
2. I can see Milvus server is up. Then try to run following python code:
from pymilvus import CollectionSchema, FieldSchema, DataType, connections
connections.connect(
alias="default",
host='localhost',
port='19530'
)
book_id = FieldSchema(
name="book_id",
dtype=DataType.INT64,
is_primary=True,
)
book_name = FieldSchema(
name="book_name",
dtype=DataType.VARCHAR,
max_length=200,
)
word_count = FieldSchema(
name="word_count",
dtype=DataType.INT64,
)
book_intro = FieldSchema(
name="book_intro",
dtype=DataType.FLOAT_VECTOR,
dim=2
)
schema = CollectionSchema(
fields=[book_id, book_name, word_count, book_intro],
description="Test book search"
)
collection_name = "book"
from pymilvus import Collection
collection = Collection(
name=collection_name,
schema=schema,
using='default',
shards_num=2
)
index_params = {
"metric_type":"L2",
"index_type":"IVF_FLAT",
"params":{"nlist":1024}
}
collection.create_index(
field_name="book_intro",
index_params=index_params
)
partition = collection.create_partition("p2")
collection.release()
collection.load(["_default"])
collection.partition("p2").release()
3. The program is blocked. I can't release the book colleciton any more unless I restart Milvus.
Milvus Log
No response
Anything else?
No response
This issue exists in both 2.2.6 and master-latest.
The following is the logs for master latest: master: master-20230423-092d7439 pymilvus: 2.4.0.dev15 logs: 23176.log
The root cause was waitCollectionReleased method block forever when releasing non-loaded partitions. A pr has been submit to solve this problem.
Patch merged for branch 2.2.0 @binbinlv could you please verify? PR for master is #23623
Patch merged for branch 2.2.0 @binbinlv could you please verify? PR for master is #23623
OK, verifying.
Verified on branch 2.2.0: 2.2.0-20230423-b54afdbb
Fixed: Releasing all partitions Released all partitions Loading the default partition Loaded the default partition releasing the unloaded partition released the unloaded partition
@shadowor1982a this issue has been fixed in 2.2.0-branch, and will release in 2.2.7 soon, thanks very much for your issues.
Verified on branch 2.2.0: 2.2.0-20230423-b54afdbb
Fixed: Releasing all partitions Released all partitions Loading the default partition Loaded the default partition releasing the unloaded partition released the unloaded partition
And will verify the master branch, when the solution pr for master merged.
@binbinlv I think patch is in latest release(v2.2.8), shall we could this issue?
@binbinlv I think patch is in latest release(v2.2.8), shall we could this issue?
close this issue?
@binbinlv I think patch is in latest release(v2.2.8), shall we could this issue?
It is verified and fixed on my side in v2.2.8.
@shadowor1982a Could you help to verify it on your side and then you could close this issue. Thanks.