scrapyrt
scrapyrt copied to clipboard
Added: Enhanced spdier log/error handling
Collects not only spider exceptions, but also other kinds of errors that appear in logs. E.g.:
{
"status": "ok",
"errors": [
"Ignoring response <404 https://www.linkedin.com/in/no_such_profile>: HTTP status code is not handled or not allowed"
],
"items": []
}
Currently for reviewing purpose only. Please do not merge until there're unit tests added.