milvus-sdk-node icon indicating copy to clipboard operation
milvus-sdk-node copied to clipboard

Search with iterator

Open BeAllAround opened this issue 10 months ago • 14 comments

Describe the feature:

It doesn't seem to be possible to "search with iterator" as documented in:

https://milvus.io/docs/v2.3.x/with_iterators.md#Search-with-iterator

Describe a specific use case for the feature:

avoiding: invalid max query result window, (offset+limit) should be in range [1, 16384] ERROR

BeAllAround avatar Mar 26 '24 18:03 BeAllAround

Do you really need this feature? if so, I will add it later.

shanghaikid avatar Mar 27 '24 06:03 shanghaikid

@shanghaikid well, if there is a workaround where I can return more than 16384 items ( if a collection contains so ) with the milvusClient.search, do let me know. Thank you very much for your consideration.

BeAllAround avatar Mar 27 '24 10:03 BeAllAround

@shanghaikid well, if there is a workaround where I can return more than 16384 items ( if a collection contains so ) with the milvusClient.search, do let me know. Thank you very much for your consideration.

I see, I will implement this feature.

shanghaikid avatar Mar 27 '24 10:03 shanghaikid

@shanghaikid Awesome - thanks. If you could republish @zilliz/milvus2-sdk-node via npm then as well, please. That'd be great. I am using package managers to install it.

BeAllAround avatar Mar 27 '24 10:03 BeAllAround

@shanghaikid Awesome - thanks. If you could republish @zilliz/milvus2-sdk-node via npm then as well, please. That'd be great. I am using package managers to install it.

Of course, but it takes time. stay tuned.

shanghaikid avatar Mar 27 '24 10:03 shanghaikid

Hi, What is the status of the issue? It would be nice to has the iterator for search and query

enricrp-rea avatar Apr 22 '24 10:04 enricrp-rea

Hi, What is the status of the issue? It would be nice to has the iterator for search and query

sorry, it's on hold,I've finished the query iterator, but based on my testing, the search iterator is not accurate, I am waiting for milvus team 's response.

shanghaikid avatar Apr 22 '24 11:04 shanghaikid

can you give me your case, I can do some extra testing specifically for your case, like what's your schema, which metric do you like to use?

shanghaikid avatar Apr 22 '24 11:04 shanghaikid

I need to get all primary keys from all the entities. So right now I am iterating through pagination until there are no more results left. However, the right solution would be to have the iterator

enricrp-rea avatar Apr 24 '24 15:04 enricrp-rea

I need to get all primary keys from all the entities. So right now I am iterating through pagination until there are no more results left. However, the right solution would be to have the iterator

so all you need is query iterator, not search right? the query iterator is ready, but search is not good.

shanghaikid avatar Apr 25 '24 01:04 shanghaikid

Yes, in my case i only need a query iterator. Thx

enricrp-rea avatar Apr 25 '24 09:04 enricrp-rea

Yes, in my case i only need a query iterator. Thx

ok, I will release one version tomorrow.

shanghaikid avatar Apr 25 '24 10:04 shanghaikid

Yes, in my case i only need a query iterator. Thx

Hi , v2.4.2 is released. please check here to view the usage method:

https://github.com/milvus-io/milvus-sdk-node/tree/v2.4.2?tab=readme-ov-file#whats-new-in-v242

shanghaikid avatar Apr 26 '24 02:04 shanghaikid

Hey thx for the quick reply and update. Just one last thing, I have seen that the limit is required but the documentation sais it is optional. Thx!!

enricrp-rea avatar Apr 26 '24 14:04 enricrp-rea