browsertrix icon indicating copy to clipboard operation
browsertrix copied to clipboard

[Feature]: Add support for custom crawl headers

Open tw4l opened this issue 1 year ago • 1 comments
trafficstars

What change would you like to see?

Requested on IIPC Slack:

"We need the option to set a request header name and value in the configuration. It could be e.g. cookie: Zxcv1234 and it should then only be set in the request headers if domain name or URL is specified else set on all URLs request. Header Name = cookie Header Value = Zxcv1234"

Context

From IIPC Slack: "What we are missing is the option to manually add multiple custom headers and cookies to a crawl configuration, it could be the same place as user-agent is now. Some hosts offer that as the only access to their content behind a paywall."

This would require adding support for custom headers to the crawler as well as the Browsertrix workflow editor.

tw4l avatar Oct 08 '24 14:10 tw4l