adstxtcrawler icon indicating copy to clipboard operation
adstxtcrawler copied to clipboard

Crawler doesn't fail if the first data line contains non-schema data

Open BrendanIAB opened this issue 8 years ago • 0 comments

Test case results in HTML file being sent to parse, generating unicode parsing error.

  • Consider verifying that the first non-comment line contains exactly 3 or 4 fields around adstxt_crawler.py line 140.

BrendanIAB avatar Jul 11 '17 16:07 BrendanIAB