ArchiveBot icon indicating copy to clipboard operation
ArchiveBot copied to clipboard

Replace hardcoded cookies with a proper cookie jar

Open JustAnotherArchivist opened this issue 6 years ago • 1 comments

This classifies as a bug because it seems to prevent other cookies from being set. For example, this breaks Blogger's "content warning" thingy, see job 7c6i9gqmp0al3kuk27vat3dyv.

JustAnotherArchivist avatar Aug 24 '19 11:08 JustAnotherArchivist

There are two ways how this could be implemented: there could be a cookie jar in the pipeline directory, or it could be pulled from the control node on starting the job. The latter would probably be better since it wouldn't require pipeline stopping and upgrading to get the new cookies for new jobs, but it's also a bit more complex to implement. If done that way, it should not use CouchDB if possible (#132).

JustAnotherArchivist avatar Aug 24 '19 11:08 JustAnotherArchivist