[Bug]: [major compaction] Failed to get compaction state when the major compaction is not completed

[binbinlv]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version: master-latest
- Deployment mode(standalone or cluster):both
- MQ type(rocksmq, pulsar or kafka):    all
- SDK version(e.g. pymilvus v2.0.0rc2): private repo
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Failed to get compaction state when the major compaction is not completed

>>> collection.compact(is_major=True)
RPC error: [compact], <MilvusException: (code=2303, message=448957204652099726: collection is compacting)>, <Time:{'RPC start': '2024-04-09 06:38:42.196615', 'RPC error': '2024-04-09 06:38:42.201401'}>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 1498, in compact
    self.compaction_id = conn.compact(self._name, timeout=timeout, is_major=is_major, **kwargs)
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/decorators.py", line 147, in handler
    raise e from e
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/decorators.py", line 143, in handler
    return func(*args, **kwargs)
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/decorators.py", line 182, in handler
    return func(self, *args, **kwargs)
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/decorators.py", line 122, in handler
    raise e from e
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/decorators.py", line 87, in handler
    return func(*args, **kwargs)
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 1531, in compact
    check_status(response.status)
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/client/utils.py", line 60, in check_status
    raise MilvusException(status.code, status.reason, status.error_code)
pymilvus.exceptions.MilvusException: <MilvusException: (code=2303, message=448957204652099726: collection is compacting)>

>>> collection.get_compaction_state()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 1512, in get_compaction_state
    return conn.get_compaction_state(self.compaction_id, timeout=timeout, **kwargs)
AttributeError: 'Collection' object has no attribute 'compaction_id'

Expected Behavior

Return the compaction state info:

CompactionState

• compaction id: *
• State: *
• executing plan number: *
• timeout plan number: 0
• complete plan number: 0

Steps To Reproduce

import os
import time
import random
import string
import numpy as np
from pymilvus import (
    connections,
    utility,
    FieldSchema, CollectionSchema, DataType,
    Collection,
)

fmt = "\n=== {:30} ===\n"
dim = 128

print(fmt.format("start connecting to Milvus"))
connections.connect("default", host=" ", port="19530")

default_fields = [
   FieldSchema(name="count", dtype=DataType.INT64, is_primary=True),
   FieldSchema(name="key", dtype=DataType.INT16),
   FieldSchema(name="random", dtype=DataType.DOUBLE),
   FieldSchema(name="var", dtype=DataType.VARCHAR, max_length=10000, is_primary=False),
   FieldSchema(name="embeddings", dtype=DataType.FLOAT_VECTOR, dim=dim, is_clustering_key=True)
]
default_schema = CollectionSchema(fields=default_fields, description="test clustering-key collection")
collection_name = "major_compaction_collection"
if utility.has_collection(collection_name):
   collection = Collection(name=collection_name)
   collection.drop()
   print("drop")
hello_milvus = Collection(name=collection_name, schema=default_schema)

nb = 10000

rng = np.random.default_rng(seed=19530)
random_data = rng.random(nb).tolist()

vec_data = [[random.random() for _ in range(dim)] for _ in range(nb)]
_len = int(20)
_str = string.ascii_letters + string.digits
_s = _str
print("_str size ", len(_str))

for i in range(int(_len / len(_str))):
    _s += _str
    print("append str ", i)
values = [''.join(random.sample(_s, _len - 1)) for _ in range(nb)]
index = 0
while index < 1024:
    # insert data
    data = [
        [index * nb + i for i in range(nb)],
        [random.randint(0, 1024) for i in range(nb)],
        random_data,
        values,
        vec_data,
    ]
    start = time.time()
    res = hello_milvus.insert(data)
    end = time.time() - start
    print("insert done in %f" % end, index)
    index += 1
    hello_milvus.flush()

print(f"Number of entities in Milvus: {hello_milvus.num_entities}")  # check the num_entites

hello_milvus.compact(is_major=True)
collection.get_compaction_state()

Milvus Log

No response

Anything else?

No response

[binbinlv]

And the same issue exists in the interface "get_compaction_plans()" and "wait_for_compaction_completed()", need check too.

>>> collection.wait_for_compaction_completed()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 1530, in wait_for_compaction_completed
    return conn.wait_for_compaction_completed(self.compaction_id, timeout=timeout, **kwargs)
AttributeError: 'Collection' object has no attribute 'compaction_id'
>>> collection.get_compaction_plans()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/major_compaction/pymilvus/virtual/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 1543, in get_compaction_plans
    return conn.get_compaction_plans(self.compaction_id, timeout=timeout, **kwargs)
AttributeError: 'Collection' object has no attribute 'compaction_id'

And the "compact" interface does not return "compaction_id" when the interface returns.

[binbinlv]

verified "wait_for_compaction_completed()" and "get_compaction_state()", they are fixed.

But "get_compaction_plans()" does not support major compaction yet.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.