cloudflare-scrape-Android
cloudflare-scrape-Android copied to clipboard
zhkrb
getting cloudflare page...
Hello not working for me :/ did i any thing wrong??
private const val USER_AGENT =
"Mozilla/5.0 (Linux; Android 6.0.1; SM-G920V Build/MMB29K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.98 Mobile Safari/537.36"
url = "https://socialblade.com/youtube/top/country/ca"
fun getResquest(context: Context, url: String) {
val cloudflare = Cloudflare(context, url)
cloudflare.user_agent = USER_AGENT
cloudflare.setCfCallback(object : CfCallback {
override fun onSuccess(
cookieList: MutableList<HttpCookie>?,
hasNewUrl: Boolean,
newUrl: String?
) {
val test = Jsoup.connect(url).cookies(ConvertUtil.List2Map(cookieList))
.ignoreHttpErrors(true).ignoreContentType(true).timeout(10000)
.userAgent(USER_AGENT).get()
test.body().toString()
}
override fun onFail(code: Int, msg: String?) {
throw RuntimeException()
}
})
cloudflare.getCookies()
}
the val test.body is returning this html:
|
|
Yes.. Not working for me too... Always onFailed
actually i got onSuccess, but the response page is cloudflare html
Looks very strange, check passed on HttpURLConnent, but same cookies in jsoup is not work, even if they have same header....
@zhkrb Ok thanks.... May i ask if rhino based is working or not.... If not why don't you remove folders
@zhkrb now its works! but now open like a webview with a captch page to complete challenger, is it normal?
just another feedback, will be cool if u let progress bar to users responsability, eg: i use shimmer to load my list and dont want that progress_bar from the lib in front every thing.... By the way, keep the good work, your lib saved my life 😄
@naveedhassan913 rhino branch is not working. Now it's hrad to parse with v8 and rhino engine, I'm trying to make them work. Because webview branch realize with code is so dirty
@lfgtavora yes, webview branch needs a webview to pass the js check. Even if the anti DDOS page does not return CAPTCHA challenge, it has a hidden webview working in dialog.
I'll add a switch to hide the dialog. But in my opinion, get cookie in all network requests is not a good idea. you should Maintaining a global cookie, and decision whether to update cookie and Re request according of the response.
excuse me @zhkrb , can you help me to bypass cloudflare from this website? (http://komikcast.com/)
I always got onFailed when I used your library that contains webView in there
implementation 'com.zhkrb.cloudflare-scrape-android:scrape-webview:0.0.3'
thank you 😁
@Abealkindy This website is looks like set up to be accessible only by US IP.
CN/RU/JP ip is always return Access denied in my test
anyone can check this site (https://komikindo.co) Now I checked with the application that the creator of the repository shared in another branch, it is processed successfully. It's really worth the wait. .
