gospider icon indicating copy to clipboard operation
gospider copied to clipboard

robots.txt dir not found

Open Darks1de42 opened this issue 1 year ago • 0 comments

Hi, I'm currently making the portswigger academy with this challenge: Infoleak via Backup files. It has a robots.txt with the following content:

User-agent: *
Disallow: /backup

Unfortunately gospider didn't find it. What is the reason for that maybe a :bug: ? My command is:

gospider -s https://0a0700200444716480a3d677000e007a.web-security-academy.net/ -o sites -d 4 --sitemap --robots  -p http://127.0.0.1:8080 -ra

Darks1de42 avatar Aug 24 '23 20:08 Darks1de42