urldedupe
urldedupe copied to clipboard
Can't remove duplicate files
jsfiles.txt
https://www.test.com/js/0-0c6e5e47ca6a3f3f7243.js
https://www.test.com/js/0-249c4f63764b90e95f29.js
https://www.test.com/js/0-356c7b1d95f2143f6cd2.js
https://www.test.com/js/0-5adfe0ed1f01b27b5f5f.js
https://www.test.com/js/0-6553d716c12f03bb710d.js
cat jsfiles.txt | urldedupe -s
expected output should be: https://www.test.com/js/0-0c6e5e47ca6a3f3f7243.js