[Bug]: [restful v2]when nq > 1, the hybrid search can return all results, but it flattens them

[zhuwenxing]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version:master/2.4
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

when nq > 1, the hybrid search can return all results, but it flattens them

[1,2,...20]

Expected Behavior

[ [1_1,...1_10], [2_1,...2_10] ]

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[zhuwenxing]

/assign @PowderLi

[PowderLi]

/assign @czs007

[PowderLi]

how to produce? @zhuwenxing could you please provide an example?

[zhuwenxing]

see https://github.com/zhuwenxing/milvus/blob/9f8e89000d8ef42aaac1b5d025059930e8898658/tests/restful_client_v2/testcases/test_vector_operations.py#L1234

[PowderLi]

what about the result of python sdk? while nq > 1, search api also return [1,2,...20], is it unexpected?

[zhuwenxing]

python sdk

    res = collection.search(
        vectors[-2:], "float_vector", search_params, topK,
        "int64 > 100", output_fields=["int64", "float"], timeout=TIMEOUT
    )
    t1 = time.time()
    print(f"search cost  {t1 - t0:.4f} seconds")
    # show result
    for hits in res:
        for hit in hits:
            # Get value of the random value field for search result
            print(hit, hit.entity.get("float"))

so, in my opinion, restful should be like following

Expected Behavior [ [1_1,...1_10], [2_1,...2_10] ]

[xiaofan-luan]

I think the different is: Python SDK can wrap on top of the proto, so it's more efficient driven. But restful need to response ona way easier to understand?

[yiwen92]

Restful response need split for nq>1. Or we will lose info in a flatterned result.(considering one return does not meet enough limit counts)

[yiwen92]

BTW Is there any proved case say we must need nq>1 for hybrid search? @xiaofan-luan

[smellthemoon]

/assign

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[zhuwenxing]

It was resolved through another approach - the returned result is still a list, but an additional topk list is also output, requiring users to perform the segmentation themselves based on the topk list.