go-sitemirror icon indicating copy to clipboard operation
go-sitemirror copied to clipboard

Add the "Referer" header to requests.

Open ndbiaw opened this issue 1 year ago • 0 comments

Problem: Many websites use the "Referer" header to protect the content on the page, causing their mirroring to fail.

Solution: Add the "Referer" header with the value of the website domain name to every request. This header will not change any content or change the server response to the request, but it will resolve hotlink blocking on most sites.

ndbiaw avatar Sep 02 '23 13:09 ndbiaw