gospider
gospider copied to clipboard
robots.txt dir not found
Hi, I'm currently making the portswigger academy with this challenge: Infoleak via Backup files. It has a robots.txt with the following content:
User-agent: *
Disallow: /backup
Unfortunately gospider didn't find it. What is the reason for that maybe a :bug: ? My command is:
gospider -s https://0a0700200444716480a3d677000e007a.web-security-academy.net/ -o sites -d 4 --sitemap --robots -p http://127.0.0.1:8080 -ra