sotoki icon indicating copy to clipboard operation
sotoki copied to clipboard

Stop relying on online archive.org Sites.xml

Open benoit74 opened this issue 4 months ago • 1 comments

Now that archive.org is down, it becomes obvious that the scraper is down as well despite the fact that we have mirrored all dumps. And the reason is a bit sad: we did not mirrored Sites.xml and we are fetching it from online ... which is not online anymore.

I think we must mirror Sites.xml as well, and use the Sites.xml for our S3 bucket.

benoit74 avatar Oct 22 '24 20:10 benoit74