shadowmoose
Suggestion for DCT Hash Image for deduplication process
Current deduplicator is not working most of the time. I tested a reddit addon for live deduplication in browser, and it works 100% of the time (I highly recommend it)
https://github.com/nickgaya/rededup
The deduplication algorithm that is being used is fast-DCT
It would be a huge improvement.
I appreciate the suggestion, but the hashing function seems to be working fine for most test cases I have. The implementation you linked to is certainly more advanced, and probably better overall, but I have concerns about how to manage storing the structures required for comparison.
Basically when it comes down to it, this approach is currently over my head and because RMD already has a functional implementation I don't have time to really dig in and figure it out. If somebody else can offer a node-side implementation of this, I'd be open to adding it.
In the rewrite of RMD, porting it to TypeScript, I have added a setting to adjust the comparison similarity cutoff RMD uses. In testing, this can be used to greatly adjust how similar images are allowed to be. Hopefully this adjustment will let you filter things more to your liking.
EDIT: To clarify, I like this suggestion a lot and would love to implement it, but cannot do it myself at this point. I'll leave this open in case anybody happens to be looking for something to contribute.
