nil0x42
Add support for removing duplicates from other file
It's often useful to be able to check If new wordlist you find is "interesting" at all. Currently duplicut can remove duplicates only in single but how about something like this: duplicut wordlist1.txt -i wordlist2.txt -o clean-wordlist1.txt It could be achieved by creating a temporary file that combines worlist1.txt and wordlist2.txt it is just important to skip first n (number of wordlist2.txt liens) in output. Rest could function the same way it does currently...
wc -l wordlist1.txt we are checking the number of lines in this document. to find out how many there were... cat wordlist1.txt wordlist2.txt > wordlist3.txt duplicut wordlist3.txt -o clean-wordlist.txt split -l #number of lines# clean-wordlist.txt output
Yeah, i like the KISS principle. Do one thing, and do it well. I also frequently find myself using duplicut in combination with other linux commands. That's the way, and you can easily automate more complex workflows like yours with little to no overhead compared to a difficult c implementation of all features.
