scrapy-pagestorage icon indicating copy to clipboard operation
scrapy-pagestorage copied to clipboard

Emitting a log message containing the request fingerprint when storing a page

Open starrify opened this issue 6 years ago • 1 comments

This is idea of a new feature.

Use case: When PAGE_STORAGE_ON_ERROR_ENABLED is enabled and there're errors encountered in a job, a user may want to check the corresponding page contents after having read the log messages. However it might sometimes be difficult to locate the very record in Scrapinghub Collections. With the request fingerprint provided in the log messages, a user may easily locate the record by querying the fingerprint value as the key.

starrify avatar Jun 27 '18 12:06 starrify