adstxtcrawler
adstxtcrawler copied to clipboard
Crawler doesn't fail if the first data line contains non-schema data
Test case results in HTML file being sent to parse, generating unicode parsing error.
- Consider verifying that the first non-comment line contains exactly 3 or 4 fields around adstxt_crawler.py line 140.