Overchan-Android icon indicating copy to clipboard operation
Overchan-Android copied to clipboard

8ch back pages

Open jacekmusial opened this issue 9 years ago • 6 comments

In 8ch.net back pages (page1.html, page2.html, ...) larger than 2 are disabled, due to save on I/O write.

So Overchan propose "catalog" option to browse threads on this imageboard.

"Catalog" option download file catalog.json, for example: 8ch.net/pol/catalog.json and fetch thread list from there.

Problem is: that catalog.json file contain threads only from page 1 and page 2. So you end up with ~20 threads, while in real there are much more threads.

What I propose is, of course, fetch thread list directly from catalog.html.

I'm writing here, because I want to ask, what tools are most suitable for that job. For http(s) request, use http-request by Kevin Sawicki?

Also, maybe I'm wrong and there is better option to fetch "real" thread list from 8ch.net imageboard?

jacekmusial avatar Dec 06 '15 21:12 jacekmusial

Problem is: that catalog.json file contain threads only from page 1 and page 2.

Sorry, I didn't check it at once.

What I propose is, of course, fetch thread list directly from catalog.html.

Right now I have no time to fix it.

I'm writing here, because I want to ask, what tools are most suitable for that job.

For example, look how it is made in krautchan module or in generic wakaba parser. https://github.com/miku-nyan/Overchan-Android/tree/master/src/nya/miku/wishmaster/chans/krautchan https://github.com/miku-nyan/Overchan-Android/blob/master/src/nya/miku/wishmaster/chans/krautchan/KrautReader.java https://github.com/miku-nyan/Overchan-Android/blob/master/src/nya/miku/wishmaster/api/util/WakabaReader.java

For http(s) request, use http-request by Kevin Sawicki?

Why? What do you mean?

miku-nyan avatar Dec 06 '15 21:12 miku-nyan

And whether it is necessary at all? As far as I know 8chan plans to switch to the new engine (infinity-next).

miku-nyan avatar Dec 06 '15 21:12 miku-nyan

Problem is: that catalog.json file contain threads only from page 1 and page 2.

Sorry, I didn't check it at once.

No problem, we all are human :smiley:

What I propose is, of course, fetch thread list directly from catalog.html.

Right now I have no time to fix it.

But I have. And I'm preparing to write it. Thanks for examples, this is what I was looking for.

For http(s) request, use http-request by Kevin Sawicki?

Why? What do you mean?

Nevermind. I forgot that you have better suited Http(s) class.

jacekmusial avatar Dec 06 '15 21:12 jacekmusial

As far as I know 8chan plans to switch to the new engine (infinity-next).

Yep, money-hungry inifnity-next. I forgot about that, sorry.

I can't find any information about when they would switch to new engine. And I have feeling that this will not be so quick (I don't have any proof for that).

jacekmusial avatar Dec 06 '15 21:12 jacekmusial

Maybe January 1, 2016, but I don't know exactly.

(from old infinity repo)

This software is DEPRECATED and UNMAINTAINED. Come January 1, 2016, no security fixes will be merged in.

miku-nyan avatar Dec 06 '15 21:12 miku-nyan

This is still a problem, and there aren't any plans to switch to Infinity any time soon. Do you think you could look into this sometime? Thanks.

TheGag96 avatar Mar 29 '16 19:03 TheGag96