remove the duplicate results

[gibran-abdillah]

when the tool has finished searching, remove the duplicate urls. if the result is less than "amount" do the search again until it reaches the inputted amount

[naufalardhani]

by default domhttpx will return output like below, -https://domain-123.com -https://domain-321.com -http://domain-789.com -https://domain-132.com

but actually if you set the output by adding --real-path it will be like below, -https://domain-123.com/about/program/5 -https://domain-321.com/news/article.php?id=1 -http://domain-789.com/watch?v=KnZ8h3MRuYg -https://domain-123.com/about/program/6