Corentin Barreau

internetarchive: fix item's metadata support

19

#### What is the purpose of this change? We want to start using rclone inside of the Internet Archive to upload files! The goal of this PR is to bring...

.. in order to avoid cgo.

Disk watcher pausing and resuming over short intervals

2

Noticed full hang of Zeno after disk watcher reported over and over "pausing" then "resuming". ```2025-02-07T02:09:25+01:00 [INFO] Disk space is sufficient, resuming the pipeline component=controler.diskWatcher 2025-02-07T02:10:40+01:00 [WARN] Low disk space,...

Low disk space limiter seems to not be working

2

When starting a Zeno with low disk space, it crawls anyway then crash because no disk space available left.

https://github.com/PuerkitoBio/goquery/wiki/Tips-and-tricks

Ebook extractor

1

Need to handle many/all ebook formats.

Discussion of HTTP/2 use in WARC 1.1

14

[This discussion about a (perfectly valid) use of Zstandard for WARCs](https://github.com/iipc/warc-specifications/issues/105) made me reflect on some longstanding problems in the web archiving community. For years, many web crawlers have failed...