maximum recursion depth exceeded

[Sunny-Day200]

Hello, I want to use it to filter similar images, whether I just need to call find_duplicates_to_remove() When the data volume is too large (18W), it prompts me to recurse beyond the maximum depth. What should I do

[tanujjain]

Few ways to resolve your problem:

1. Split your image corpus into several smaller directories and run find_duplicates_to_remove on each directory. You can then remove duplicates from each of these directories and then merge the leftover images in new directories and keep on repeating the process.
2. Your error gets triggered while constructing the bktree. To get rid of the necessity to build a bktree, you can first use encode_images method to get hashes of each image and then use find_duplicates with search_method set to string 'brute_force'. This will return dictionary as the output expected of find_duplicates but will run much slower. You would then need to write a custom logic to remove duplicates obtained from the dictionary. Refer documentation for the same: https://idealo.github.io/imagededup/user_guide/finding_duplicates/
3. Use mac or linux instead of Windows: With mac/linux, the default search method will be a quick version of brute force search. So, bktree will not be constructed at all.

[tanujjain]

@Sunny-Day200 Unless you have any further comments, I would be closing the issue shortly.