cookie_maker and redirects

[nlevitt]

trafficstars

pywb.utils.loaders.HttpLoader.load() sets the value of headers['Cookie'] which is used in the call self.session.get(url, headers=headers, stream=True).

When cookies are set this way, if the url redirects, the cookies are not sent to the redirect destination. This is a problem loading warcs from https://archive.org/download/... urls requiring authorization, for example.

There are other ways to set cookies such that they are sent to the redirect destination. One way is to pass the cookies parameter to session.get. Something like this:

    r = self.session.get(url, headers=headers, stream=True, cookies=self.cookie_maker)

Unfortunately this cookies parameter needs to be a dictionary, whereas pywb's configurable cookie_maker parameter is a string or a function. Not sure how best to make the change.

[Serkan-devel]

Can one then be able to create archives, while authenticated?