duplicut icon indicating copy to clipboard operation
duplicut copied to clipboard

Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)

Results 13 duplicut issues
Sort by recently updated
recently updated
newest added

Can this program sort the password dictionary?

`MEDIUM_LINE_BYTES` is currently hardcorded in `const.h`, to a value of 8. The hasmap & chunks chunks are then made in such way that if real medium length of lines is...

enhancement
perf impacted
important

1. Support for taking input from multiple input files. Wordlist can be spread across multiple files. Currently I am merging it and then passing to duplicut. Need to work something...

### HDD vs SSD On HDD, sequential access is relatively fast, while random access is terribly slow. That's why `duplicut`, written back in 2014 has been optimized thinking of it....

Hi Using the same technique as for duplicates, is it possible to remove words that are a certain word length. (min/max word length) Also if you could use the same...

enhancement

it is usefull when duplicut is run using stdin as input file.

enhancement

splice() method could be implemented too, if it works on more platforms REFERENCE: http://blog.superpat.com/2010/06/01/zero-copy-in-linux-with-sendfile-and-splice/

the SIGINT unix signal should be catched to properly clean things such as temporary created file

bug
enhancement

It's often useful to be able to check If new wordlist you find is "interesting" at all. Currently duplicut can remove duplicates only in single but how about something like...

One question... Why any line longer than 255 chars is ignored? There is any reasonable explanation? Regards.