[Bug]: [Nightly] Compact failed when initing 3 segments with same channel

[NicoYuan1986]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version:897ed62
- Deployment mode(standalone or cluster):standalone
- MQ type(rocksmq, pulsar or kafka):    rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2):2.4.0.dev12
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Compact failed when initing 3 segments with same channel.

[2023-04-22T21:44:43.354Z] >       assert len(segment_info) == 1
[2023-04-22T21:44:43.354Z] E       assert 3 == 1
[2023-04-22T21:44:43.354Z] E        +  where 3 = len([segmentID: 440974994782579250\ncollectionID: 440974994782579241\npartitionID: 440974994782579242\nnum_rows: 100\nstate: Sealed\nnodeIds: 1\n, segmentID: 440974994782579341\ncollectionID: 440974994782579241\npartitionID: 440974994782579242\nnum_rows: 100\nstate: Sealed\nnodeIds: 1\n, segmentID: 440974994782579358\ncollectionID: 440974994782579241\npartitionID: 440974994782579242\nnum_rows: 100\nstate: Sealed\nnodeIds: 1\n])
[2023-04-22T21:44:43.354Z] 

Expected Behavior

compact successfully

Steps To Reproduce

@pytest.mark.tags(CaseLabel.L1)
def test_count_compact_merge(self):
    """
    target: test count after compact merge segments
    method: 1. init 2 segments with same channel
            2. compact
            3. count
    expected: verify count
    """
    collection_w = self.init_collection_wrap(name=cf.gen_unique_str(prefix), shards_num=1)

    # init compact_segment_num_threshold segments
    tmp_nb = 100
    for i in range(ct.compact_segment_num_threshold):
        df = cf.gen_default_dataframe_data(nb=tmp_nb, start=i * tmp_nb)
        collection_w.insert(df)
        collection_w.flush()

    collection_w.create_index(ct.default_float_vec_field_name, ct.default_index)
    collection_w.compact()
    collection_w.wait_for_compaction_completed()

    collection_w.load()
    segment_info, _ = self.utility_wrap.get_query_segment_info(collection_w.name)
    assert len(segment_info) == 1

    # count after compact
    collection_w.query(expr=default_expr, output_fields=[ct.default_count_output],
                       check_task=CheckTasks.check_query_results,
                       check_items={exp_res: [{count: tmp_nb * ct.compact_segment_num_threshold}]})

Milvus Log

1. link: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI/detail/master/350/pipeline/123/
2. log: artifacts-milvus-standalone-nightly-350-pymilvus-e2e-logs.tar.gz
3. collection name: query_JDL5Gmui
4. failed time: [2023-04-22T18:32:26.375Z] [gw1] [ 31%] FAILED testcases/test_query.py::TestQueryCount::test_count_compact_merge

Anything else?

No response

[yanliang567]

/assign @XuanYang-cn /unassign

[sunby]

/assign

[NicoYuan1986]

Latest:

[2023-04-25T21:22:50.524Z] >       assert len(segment_info) == 1
[2023-04-25T21:22:50.524Z] E       assert 0 == 1
[2023-04-25T21:22:50.524Z] E        +  where 0 = len([])

1. link: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI/detail/master/353/pipeline/155
2. log: artifacts-milvus-distributed-kafka-nightly-353-pymilvus-e2e-logs.tar.gz
3. collection name: query_umbS38pF
4. failed time: [2023-04-25T18:32:20.200Z] [gw1] [ 30%] FAILED testcases/test_query.py::TestQueryCount::test_count_compact_merge

[sunby]

@NicoYuan1986 What is the value of compact_segment_num_threshold?

[NicoYuan1986]

@NicoYuan1986 What is the value of compact_segment_num_threshold?

It's 3.

[sunby]

@NicoYuan1986 If the new generated segment's index is not built, load will use old segments.

[NicoYuan1986]

Not reproduced.