8ch back pages

[jacekmusial]

In 8ch.net back pages (page1.html, page2.html, ...) larger than 2 are disabled, due to save on I/O write.

So Overchan propose "catalog" option to browse threads on this imageboard.

"Catalog" option download file catalog.json, for example: 8ch.net/pol/catalog.json and fetch thread list from there.

Problem is: that catalog.json file contain threads only from page 1 and page 2. So you end up with ~20 threads, while in real there are much more threads.

What I propose is, of course, fetch thread list directly from catalog.html.

I'm writing here, because I want to ask, what tools are most suitable for that job. For http(s) request, use http-request by Kevin Sawicki?

Also, maybe I'm wrong and there is better option to fetch "real" thread list from 8ch.net imageboard?