scrapy-sentry
scrapy-sentry copied to clipboard
Capture exceptions outside of spiders
scrapy-sentry
currently does not capture exceptions that happen outside the spider's on_error
signal. This limits the usefulness of the extension. Luckily, It is pretty easy to fix as well. I currently have this in my Scrapy settings.py
instead of configuring the extension:
from twisted.python import log
from raven import Client
client = Client(dsn=SENTRY_DSN)
def log_sentry(dictionary):
if dictionary.get('isError') and 'failure' in dictionary:
try:
# Raise the failure
dictionary['failure'].raiseException()
except:
# so we can capture it here.
client.captureException()
log.addObserver(log_sentry)
It should be pretty easy to do the same in the extension, and possibly remove the on_error
handling. @llonchj, if you agree, I'm happy to prepare a PR for that.
Above snippet is not my work, it was discovered by @elmkarami in http://stackoverflow.com/questions/25262765/handle-all-exception-in-scrapy-with-sentry and https://github.com/scrapy/scrapy/issues/852.