[BUG] Cannot bypass cloudflare

[jsanchezflores]

Describe the bug [ERROR]Service Unavailable! (503)

Cannot download from foxaholic because novel-grabber/in-app browser Chrome (tried others also) cannot bypass cloudflare. I can access Foxaholic just fine with my regular browser (Chrome).

Expected behavior It seems to have happened recently because I was able to download with no issues before.

Platform Windows, Java 8

Logs

[ERROR]Could not establish connection to SMTP Server. Check email settings and restart.
[INFO]Checking World?s Fastest Level up! ~as a Result of the Awakening of My Incompetence Skill [Transfer Within Dungeon], I Became the Strongest Without Being Restricted by the Rules of the Dungeon.
[ERROR]Service Unavailable! (503)
[INFO]Checking A Banished Failure Survives in the Borderland and Becomes an S-Rank Exorcist
[ERROR]Service Unavailable! (503)
[INFO]Checking Disciple of Immortal
[ERROR]Service Unavailable! (503)
[INFO]Checking I Became the Strongest With The Failure Frame?Abnormal State Skill?As I Devastated Everything
[ERROR]Service Unavailable! (503)

[Flameish]

Nothing I can do about that, I searched long for a solution but haven't found one yet. You can try at a later time again, Cloudflare is often only temporary.

[Flameish]

Keeping it open for visibility.

[lunevale]

Just wondering, would NG launcher work if I used something like https://github.com/VeNoMouS/cloudscraper to bypass the cloudflare on foxaholic first?

[jsanchezflores]

Foxaholic removed cloudflare protection for the time being. It's been working for me fine since then.

[lunevale]

@jsanchezflores Thanks for letting me know! It's working for me now too. How do you know when they've removed cloudflare protection though?

[jsanchezflores]

I use the website often and the cloudflare protection wasn't checking with my browser anymore, also remember to add wait time of at least 10 (10000) sec when downloading novels. This is one of the reasons why they even add protection in the first place to prevent this.

[thornt5748]

Currently trying to use it now, but receiving an error message [ERROR] Could not connect to webpage!

Steps take: Launch NG-Launcher.jar Entered Novel URL: https://www.foxaholic.com/novel/i-was-caught-up-in-a-hero-summoning-but-that-world-is-at-peace/ Clicked "Check availability of Novel"

[frykauf]

Just wondering, would NG launcher work if I used something like https://github.com/VeNoMouS/cloudscraper to bypass the cloudflare on foxaholic first?

Or what Jdownloader does, just making the automation launch the browser with specified/default profile, that should include all the cookies for novelupdates and other sites as long as you completed the captcha once before and it saves to that browser profile’s cookies.

[Flameish]

Cloudflare is pretty good at detecting automated browser activity. Cookies won't matter in that case.

[frykauf]

Then maybe separate hCaptcha manual solver either in set browser or in app pop up window? Some downloaders do that. JDownloader does it through the separate manual solver & browser extension to pass the solve to Jdownloader & some downloaders do it via pop up browser window through internal headless browser - there you manually solve the captcha without browser/browser extension.

I do wonder if by this point if it wouldn’t be easier to make Novelupdates Jdownloader plugin, that passes the data to Novel Grabber.