python-scraperlib icon indicating copy to clipboard operation
python-scraperlib copied to clipboard

Add utility function to confirm ZIM can be created at given location / name

Open benoit74 opened this issue 1 year ago • 0 comments

In warc2zim, we have a tooling which confirms that:

  • output folder exists
  • output folder is writable
  • ZIM name is acceptable by current filesystem

This is run at the beginning of the scraper, to avoid loosing time scraping stuff and then failing many hours later.

https://github.com/openzim/warc2zim/blob/32d1a20e5df425ed737f07d91155978fe0a92bed/src/warc2zim/converter.py#L127-L162

Sharing this logic in zimscraperlib would be great since all scrapers should do these checks indeed.

benoit74 avatar May 21 '24 07:05 benoit74