recursebuster icon indicating copy to clipboard operation
recursebuster copied to clipboard

Published 20 hours ago • verified publisher icon C-Sto

url.Parse before send to channel

Open theblackturtle opened this issue 4 years ago • 1 comments

Hi, I think you should add url.Parse for url parse from html before send it to channel because something like this can happen.

GOOD: GET Found https://xxx.io/www.xxx.com --><!-- Last Published: Wed Sep 25 2019 00:19:19 GMT+0000 (UTC) --><html data-wf-domain="xxx.io" data-wf-page="5c6f4e9e89c36802e87289f8" data-wf-site="5c6eefaaeddf9248ac13bc72"><head><meta charset="utf-8"/phpmyadmin2017%2F [301 Moved Permanently] Length: 182 http://xxx.io/www.xxx.com%20--><!--%20Last%20Published:%20Wed%20Sep%2025%202019%2000:19:19%20GMT+0000%20(UTC)%20--><html%20data-wf-domain="xxx.io"%20data-wf-page="5c6f4e9e89c36802e87289f8"%20data-wf-site="5c6eefaaeddf9248ac13bc72"><head><meta%20charset="utf-8"/phpmyadmin2017

theblackturtle avatar Oct 19 '19 04:10 theblackturtle

This looks like it's probably more of a bug in the spider function which I've never really been super happy with (https://github.com/C-Sto/recursebuster/blob/0a43a54f2b61db9a11334d21ecce634be19145b7/pkg/net/net.go#L241). Adding an url.Parse will just hide that buggy code rather than resolving it (but there should definitely be an url.Parse as well)

C-Sto avatar Oct 22 '19 10:10 C-Sto