[Bug]: [Pagination] Search pagination binary sometimes got incorrect result

[NicoYuan1986]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version:42d371f
- Deployment mode(standalone or cluster):standalone
- SDK version(e.g. pymilvus v2.0.0rc2):2.2.0.dev32
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Search pagination binary sometimes got incorrect result.

>       assert search_res[0].ids == res[0].ids[offset:]
E       assert [159, 2536, 2...46, 495, 1501] == [159, 1704, 2...825, 276, ...]
E         At index 1 diff: 2536 != 1704
E         Full diff:
E         - [159, 1704, 2536, 2298, 825, 276, 534, 446, 495, 1501]
E         ?             ------------
E         + [159, 2536, 2298, 1704, 825, 276, 534, 446, 495, 1501]
E         ?       ++++++++++++

Expected Behavior

Succeed.

Steps To Reproduce

The case:

    @pytest.mark.tags(CaseLabel.L1)
    def test_search_binary_with_pagination(self, offset, auto_id):
        """
        target: test search binary with pagination
        method: create connection, collection, insert and search
        expected: search successfully
        """
        # 1. initialize with data
        collection_w, _, _, insert_ids = \
            self.init_collection_general(prefix, True, is_binary=True, auto_id=auto_id, dim=default_dim)[0:4]
        # 2. search
        search_param = {"metric_type": "JACCARD", "params": {"nprobe": 10}, "offset": offset}
        binary_vectors = cf.gen_binary_vectors(default_nq, default_dim)[1]
        search_res = collection_w.search(binary_vectors[:default_nq], "binary_vector",
                                         search_param, default_limit,
                                         check_task=CheckTasks.check_search_results,
                                         check_items={"nq": default_nq,
                                                      "ids": insert_ids,
                                                      "limit": default_limit})[0]
        search_binary_param = {"metric_type": "JACCARD", "params": {"nprobe": 10}}
        res = collection_w.search(binary_vectors[:default_nq], "binary_vector", search_binary_param,
                                  default_limit + offset)[0]
        assert search_res[0].ids == res[0].ids[offset:]

Milvus Log

No response

Anything else?

No response