[FEATURE]: Is there a solution about imae duplication?

[GongZhengLi]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is your feature request related to a problem? Please describe.

我想对亿级的图片数据进行去重

Describe the solution you'd like

能否之间基于milvus实现 ？

Describe alternatives you've considered

No response

Anything else?

No response

[shiyu22]

@GongZhengLi I think you can try Towhee, and there is an Example about image depulication.

And you can resolve the large-scale data with Milvus. You can refer to reverse-image-search solution, and you can set the threshold of search results to filter out duplicate images like in towhee-example.