warcreate icon indicating copy to clipboard operation
warcreate copied to clipboard
verified publisher icon machawk1

Robots.txt record is not included in the WARC record

Open machawk1 opened this issue 12 years ago • 0 comments

It's questionable whether this should even be included but for the sake of comprehensive preservation maybe it should be fetched

machawk1 avatar May 23 '13 13:05 machawk1