scrapy-pagestorage

scrapy-pagestorage copied to clipboard

ValueTooLarge: Value exceeds max encoded size of 1048576 bytes

2

the issue arises when there are null characters ( \x00) present in the response.body for len(response.body) == every null characters will be of length 1 but when we are encoding...

DharmeshPandav

Emitting a log message containing the request fingerprint when storing a page

1

This is idea of a new feature. Use case: When `PAGE_STORAGE_ON_ERROR_ENABLED` is enabled and there're errors encountered in a job, a user may want to check the corresponding page contents...

starrify