imgbrd-grabber icon indicating copy to clipboard operation
imgbrd-grabber copied to clipboard

rule34hentai no longer working

Open ayoits0913 opened this issue 5 years ago • 22 comments

That site has always worked great with this program but as of today, I cannot connect to it. It just says there is no result

ayoits0913 avatar Dec 31 '19 09:12 ayoits0913

Hi, I would think that your page number is incorrect. It should be on first page when u enter new search.

On 31 Dec 2019, at 10:38, ayoits0913 <[email protected]mailto:[email protected]> wrote:

That site has always worked great with this program but as of today, I cannot connect to it. It just says there is no result

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/Bionus/imgbrd-grabber/issues/1858?email_source=notifications&email_token=ANLR6JMB45Z7NR5XCASO5BTQ3MHIVA5CNFSM4KBUDJAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IDO53TQ, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANLR6JNZ3OOFGWR7EYELWPLQ3MHIVANCNFSM4KBUDJAA.

HawkSoul avatar Dec 31 '19 11:12 HawkSoul

It is on the first page. Even after deleting the appdata folder and doing a fresh install, it isn't working.

I can connect to the site fine with my browser, and even test the login info on the program, and it says "success".. yet it still says "No valid source of the site returned results." I've been checking the site on this program every day for months, very strange.

ayoits0913 avatar Jan 01 '20 03:01 ayoits0913

This is from the log:

[22:40:46.731][Info] Loading results... [22:40:46.731][Info] [rule34hentai.net][Html] Loading page https://rule34hentai.net/post/list/1 [22:40:46.992][Info] [rule34hentai.net][Html] Receiving page https://rule34hentai.net/post/list/1 [22:40:46.992][Error] [rule34hentai.net][Html] Loading error: Error transferring https://rule34hentai.net/post/list/1 - server replied: Shimmie (203) [22:40:46.993][Warning] [rule34hentai.net] Loading using Html failed. Retry using Rss. [22:40:46.993][Info] [rule34hentai.net][Rss] Loading page https://rule34hentai.net/rss/images/1 [22:40:47.176][Info] [rule34hentai.net][Rss] Receiving page https://rule34hentai.net/rss/images/1 [22:40:47.176][Error] [rule34hentai.net][Rss] Loading error: Error transferring https://rule34hentai.net/rss/images/1 - server replied: Shimmie (203) [22:40:47.176][Warning] [rule34hentai.net] No valid source of the site returned result.

ayoits0913 avatar Jan 01 '20 03:01 ayoits0913

Hi First of all Happy new year and best wishes to all of you!

And second I honestly never used that site, but is there a possibility that the https:// giving trouble? Try to change the url to www..... maybe? Good luck.

On 1 Jan 2020, at 04:41, ayoits0913 <[email protected]mailto:[email protected]> wrote:

This is from the log:

[22:40:46.731][Info] Loading results... [22:40:46.731][Info] [rule34hentai.nethttp://rule34hentai.net][Html] Loading page https://rule34hentai.net/post/list/1 [22:40:46.992][Info] [rule34hentai.nethttp://rule34hentai.net][Html] Receiving page https://rule34hentai.net/post/list/1 [22:40:46.992][Error] [rule34hentai.nethttp://rule34hentai.net][Html] Loading error: Error transferring https://rule34hentai.net/post/list/1 - server replied: Shimmie (203) [22:40:46.993][Warning] [rule34hentai.nethttp://rule34hentai.net] Loading using Html failed. Retry using Rss. [22:40:46.993][Info] [rule34hentai.nethttp://rule34hentai.net][Rss] Loading page https://rule34hentai.net/rss/images/1 [22:40:47.176][Info] [rule34hentai.nethttp://rule34hentai.net][Rss] Receiving page https://rule34hentai.net/rss/images/1 [22:40:47.176][Error] [rule34hentai.nethttp://rule34hentai.net][Rss] Loading error: Error transferring https://rule34hentai.net/rss/images/1 - server replied: Shimmie (203) [22:40:47.176][Warning] [rule34hentai.nethttp://rule34hentai.net] No valid source of the site returned result.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/Bionus/imgbrd-grabber/issues/1858?email_source=notifications&email_token=ANLR6JOOGKJOCLUNUTDJSH3Q3QGFVA5CNFSM4KBUDJAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH44RQY#issuecomment-570017987, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANLR6JJVALXJZI3UBMNETNTQ3QGFVANCNFSM4KBUDJAA.

HawkSoul avatar Jan 01 '20 10:01 HawkSoul

Happy New Year.

Unfortunately, that made no difference. Not sure what to do.

ayoits0913 avatar Jan 01 '20 16:01 ayoits0913

If anyone can test the site and let me know if it's just on my end or not, I'd appreciate it. This issue makes no sense to me, especially since I can successfully test the login info and use the site with my browser.

ayoits0913 avatar Jan 03 '20 04:01 ayoits0913

I'm having the same issue for a few days already :/ except the login test will result in failure for me, and it's working when I'm using a browser...

edit : update, I can now login sucessfully but still no result.

trucman avatar Jan 03 '20 10:01 trucman

I just did the test and this showed up: [image1.jpeg] So i chose Yes and images just loaded fine. The website seem a little different “rule34.xxx.”?

Hopefully that helps.

Cheers

On 3 Jan 2020, at 11:45, trucman <[email protected]mailto:[email protected]> wrote:

I'm having the same issue for a few days already :/ except the login test will result in failure for me, and it's working when I'm using a browser...

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/Bionus/imgbrd-grabber/issues/1858?email_source=notifications&email_token=ANLR6JMV6SY6QPPGQ33KOMTQ34JNLA5CNFSM4KBUDJAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIA3KZA#issuecomment-570537316, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANLR6JNWDRJLKGNQE3DAW5LQ34JNLANCNFSM4KBUDJAA.

HawkSoul avatar Jan 03 '20 17:01 HawkSoul

rule34.xxx is an entirely different website. That works fine for me as well. rule34hentai.net you have to add manually (not included in the sources), can you test it?

ayoits0913 avatar Jan 03 '20 21:01 ayoits0913

Same issues here, the website is completely inoperable in Grabber. Real shame, it's been one of my main sources.

jkaryan avatar Jan 06 '20 11:01 jkaryan

Same here, and i cant even open it in my browser. Gets stuck on anti-Ddos attack check

On 6 Jan 2020, at 12:08, jkaryan <[email protected]mailto:[email protected]> wrote:

Same issues here, the website is completely inoperable in Grabber. Real shame, it's been one of my main sources.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/Bionus/imgbrd-grabber/issues/1858?email_source=notifications&email_token=ANLR6JNJ4ZQZDACNYMBTZDLQ4MGI7A5CNFSM4KBUDJAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIFETSY#issuecomment-571099595, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANLR6JOSETXRHFYEBXZJTELQ4MGI7ANCNFSM4KBUDJAA.

HawkSoul avatar Jan 06 '20 12:01 HawkSoul

Same here, and i cant even open it in my browser. Gets stuck on anti-Ddos attack check

That one seems to have been fixed/come and go. It didn't work for me yesterday, does work for me today.

jkaryan avatar Jan 06 '20 15:01 jkaryan

I hope Bionus checks this issue out, this is such a great program and that's my favorite site.

ayoits0913 avatar Jan 08 '20 01:01 ayoits0913

This is all a CloudFlare issue, admin specifically set this up to stop bots and other downloaders from accessing the site.

Best way to fix this would be a headless browser for example that could approve the gated entry. Hydrus is having the same issues at this moment.

Sad to see the site that has so much exclusive content not mirrored elsewhere. All gated off from download clients, filled with ads and limited tags. Seems like admin struggles with having a grasp on the site.

Hopefully @Bionus can take a look at it and maybe find a good and simple option. The Hydrus guys will check in here from time to time for re-useable results hopefully. (need a full redesign for a headless browser)

CuddleBear92 avatar Jan 08 '20 22:01 CuddleBear92

Indeed, the owner of the website set up CloudFlare on their website, which blocks the entry of programs such as Grabber. Usually this can be circumvented by loading the page in a browser then copying the CloudFlare cookies to Grabber (IIRC it's _cfduid), but note that I haven't tested it here (I can't even access the website).

Best way to fix this would be a headless browser for example that could approve the gated entry.

Hopefully @Bionus can take a look at it and maybe find a good and simple option. The Hydrus guys will check in here from time to time for re-useable results hopefully. (need a full redesign for a headless browser)

As you said, the solution is to load the website in a headless browser (same thing for a few websites that for example use React without server-side rendering), then get the results there. But just like Hydrus, that would require a huge re-write in Grabber.

Bionus avatar Jan 10 '20 15:01 Bionus

I just tried adding that cookie, didn't change anything unfortunately.

ayoits0913 avatar Jan 10 '20 23:01 ayoits0913

No sadly just the cookie wouldn't help. There is an 1 hour access before re-verifying the access going on now from what i gather from general usage in browser.

Really hate this as R34H.net has so many 3D rendered releases and other paygated releases compared to others and does so with earlier uploads too.

It was a good decent site in terms of pure content and its upload consistency if you enjoy the content uploaded. Sadly the admin, ads, cloudflare and lacking tags detract from the site as a whole.

Sad to see the site get locked down like this as the admin uploads/scraping scripts seems to be on point. Direct from the sources of all the creators at raw quality. Something no other booru does for 3D rendered stuff atleast.

Someone in the Hydrus Discord did link to this though, some have stated that it works well with cfscrape (linked bellow)

This will only be the start sadly, its only a matter of time before more fall like this, Sankaku is prob next with how they thread request timers and other clients.

Relevant links: https://github.com/venomous/cloudscraper https://github.com/Anorov/cloudflare-scrape

CuddleBear92 avatar Jan 12 '20 03:01 CuddleBear92

You know, I just came in to talk about this exact issue after redownloading grabber to a new computer and getting set up again, and then I read the thread. God-damnit, God fucking dammit, probably the second worst shit of the new year I've had to come to.

SultrySamthepennanceman avatar Jan 28 '20 01:01 SultrySamthepennanceman

You know, I just came in to talk about this exact issue after redownloading grabber to a new computer and getting set up again, and then I read the thread. God-damnit, God fucking dammit, probably the second worst shit of the new year I've had to come to.

I feel you man. It's a big loss. I'm not about to download one image at a time, some tags have thousands of images. This program made it so easy.

ayoits0913 avatar Jan 31 '20 05:01 ayoits0913

So just coming back to this thread, I'm having certain queries go through and others not. They just hang with no error. Is this the cloudflare blocking grabber?

Spyridion avatar Jun 21 '20 03:06 Spyridion

@Spyridion The admin of the site did setup Cloudflare for its site. On the Hydrus side we have seen the cloudscraper i linked last year have worked a bit. But the dev of that have paygated some features so far, adding the 2captcha api to the parser is prob needed in the end.

We have also seen this mostly be an issue outside of the US, if you VPN to the US then you SHOULD be good aswell.

CuddleBear92 avatar Jul 31 '20 16:07 CuddleBear92

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

If this issue is about a bug that still happens in the latest version, or a suggestion that is still relevant, feel free to comment on it and the maintainers will have another look, they might have missed it!

Thank you!

stale[bot] avatar Jul 31 '22 23:07 stale[bot]