scrapex icon indicating copy to clipboard operation
scrapex copied to clipboard

Set last-modified and etag headers on request

Open sntran opened this issue 10 years ago • 1 comments

So target server can send 304 if applicable.

sntran avatar Jan 09 '16 03:01 sntran

Also, keep a hash of the content. That way, when receiving 200 response, we can do a quick check for change for servers that don't support etag. This reduces number of results that need to be parsed.

sntran avatar Jan 09 '16 03:01 sntran