ArchiveBot icon indicating copy to clipboard operation
ArchiveBot copied to clipboard

Ability to add --no-offsite-links mid grab

Open HarryC145 opened this issue 9 years ago • 1 comments

Would it be possible to add this mid grab (so say if the grab is running out of control/getting spam sites etc)

5vqqwfgciq4y7ksb6e4olk5ao is a key example of when this could be done.

HarryC145 avatar Jan 13 '16 18:01 HarryC145

This is controlled by a wpull command-line switch (see https://github.com/ArchiveTeam/ArchiveBot/blob/master/pipeline/archivebot/seesaw/wpull.py#L83-L86).

As far as I know, this is not under wpull script control, so modifications to wpull will be necessary. Paging @chfoo.

hannahwhy avatar Jan 13 '16 18:01 hannahwhy