jadegrave

Wow! Thank you for the quick response. For clarification, those are the links provided in the book…I’m using the version that is available through Safari Online Bookshelf Thanks, Jodi From:...

Thank you! Very helpful. Do you have one for the robots.txt? From: Nilesh Pandey [mailto:[email protected]] Sent: Thursday, December 21, 2017 5:02 PM To: kjam/wswp Cc: jadegrave; Author Subject: Re: [kjam/wswp]...

I see that https://webscraping.com has a robots.txt, but it doesn't have the 'Bad crawler' and other items in it.