scrapy-pagestorage
scrapy-pagestorage copied to clipboard
Emitting a log message containing the request fingerprint when storing a page
This is idea of a new feature.
Use case:
When PAGE_STORAGE_ON_ERROR_ENABLED
is enabled and there're errors encountered in a job, a user may want to check the corresponding page contents after having read the log messages.
However it might sometimes be difficult to locate the very record in Scrapinghub Collections. With the request fingerprint provided in the log messages, a user may easily locate the record by querying the fingerprint value as the key.