sarperavci
`Error checking page title: 'NoneType' object has no attribute 'title'`
Hi. I'm getting intermittent Error - 500 (Internal server error) from the /html endpoint, and the Docker logs saying Error checking page title: 'NoneType' object has no attribute 'title'. I haven't been able to isolate what exactly is triggering it, but I did notice that it happens more often in multiple pages are requested in close proximity. Below is a log of my application requesting 3 different URLs and the error occurring:
cloudflare-bypass | INFO: Started server process [1]
cloudflare-bypass | INFO: Waiting for application startup.
cloudflare-bypass | [19:08:40] INFO - cf_bypasser.server.routes - Starting Cloudflare Bypasser Server...
cloudflare-bypass | [19:08:40] INFO - cf_bypasser.server.routes - Server initialization complete
cloudflare-bypass | INFO: Application startup complete.
cloudflare-bypass | INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
cloudflare-bypass | [19:08:54] INFO - cf_bypasser.server.routes - Getting HTML content for https://TARGET_WEBSITE_1 (retries: 5, proxy: no)
cloudflare-bypass | [19:08:54] INFO - root - Getting HTML content for https://TARGET_WEBSITE_1...
cloudflare-bypass | [19:08:54] INFO - root - Using OS: windows
cloudflare-bypass | [19:08:54] INFO - root - Generated config with UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:144.0) Gecko/20100101 Firefox/144.0
cloudflare-bypass | [19:08:54] INFO - root - Screen resolution: 1680x1050
cloudflare-bypass | [19:08:54] INFO - cf_bypasser.server.routes - Getting HTML content for https://TARGET_WEBSITE_2 (retries: 5, proxy: no)
cloudflare-bypass | [19:08:54] INFO - root - Getting HTML content for https://TARGET_WEBSITE_2...
cloudflare-bypass | [19:08:54] INFO - root - Using OS: linux
cloudflare-bypass | [19:08:54] INFO - root - Generated config with UA: Mozilla/5.0 (X11; openSUSE; Linux x86_64; rv:141.0) Gecko/20100101 Firefox/141.0
cloudflare-bypass | [19:08:54] INFO - root - Screen resolution: 1920x1080
cloudflare-bypass | /usr/lib/python3.13/concurrent/futures/thread.py:59: LeakWarning: Because you did not pass in a locale region, Camoufox will generate one for you. This can cause suspicion if your IP does not match your locale region.
cloudflare-bypass | result = self.fn(*self.args, **self.kwargs)
cloudflare-bypass | /usr/lib/python3.13/concurrent/futures/thread.py:59: LeakWarning: Because you did not pass in a locale region, Camoufox will generate one for you. This can cause suspicion if your IP does not match your locale region.
cloudflare-bypass | result = self.fn(*self.args, **self.kwargs)
cloudflare-bypass | [19:09:01] INFO - root - Navigating to https://TARGET_WEBSITE_2
cloudflare-bypass | [19:09:04] INFO - root - Navigating to https://TARGET_WEBSITE_1
cloudflare-bypass | [19:09:11] INFO - root - No Cloudflare challenge detected or already bypassed
cloudflare-bypass | [19:09:11] INFO - root - Cached cookies for d41d8cd98f00b204e9800998ecf8427e, expires at 2025-11-17 21:09:11.745805
cloudflare-bypass | [19:09:13] INFO - cf_bypasser.server.routes - Successfully generated HTML content (130147 chars) and 1 cookies in 19759ms
cloudflare-bypass | [19:09:13] INFO - cf_bypasser.server.routes - Cloudflare cookies: ['cf_clearance']
cloudflare-bypass | INFO: 172.16.1.1:35288 - "GET /html?url=https://TARGET_WEBSITE_2 HTTP/1.1" 200 OK
cloudflare-bypass | [19:09:14] INFO - cf_bypasser.server.routes - Getting HTML content for https://TARGET_WEBSITE_3 (retries: 5, proxy: no)
cloudflare-bypass | [19:09:14] INFO - root - Using cached cookies for d41d8cd98f00b204e9800998ecf8427e
cloudflare-bypass | [19:09:14] INFO - root - Getting HTML content for https://TARGET_WEBSITE_3...
cloudflare-bypass | [19:09:14] INFO - root - Using OS: windows
cloudflare-bypass | [19:09:14] INFO - root - Generated config with UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:143.0) Gecko/20100101 Firefox/143.0
cloudflare-bypass | [19:09:14] INFO - root - Screen resolution: 1536x864
cloudflare-bypass | [19:09:14] INFO - root - Error checking page title: 'NoneType' object has no attribute 'title'
cloudflare-bypass | [19:09:14] INFO - root - Cloudflare challenge detected. Attempting to solve...
cloudflare-bypass | [19:09:14] INFO - root - Error determining challenge type: 'NoneType' object has no attribute 'content'
cloudflare-bypass | [19:09:14] INFO - root - Could not determine challenge type
cloudflare-bypass | INFO: 172.16.1.1:35272 - "GET /html?url=https://TARGET_WEBSITE_1 HTTP/1.1" 500 Internal Server Error
cloudflare-bypass | /usr/lib/python3.13/concurrent/futures/thread.py:59: LeakWarning: Because you did not pass in a locale region, Camoufox will generate one for you. This can cause suspicion if your IP does not match your locale region.
cloudflare-bypass | result = self.fn(*self.args, **self.kwargs)
cloudflare-bypass | [19:09:21] INFO - root - Navigating to https://TARGET_WEBSITE_3
cloudflare-bypass | [19:09:31] INFO - root - No Cloudflare challenge detected or already bypassed
cloudflare-bypass | [19:09:31] INFO - root - Cached cookies for d41d8cd98f00b204e9800998ecf8427e, expires at 2025-11-17 21:09:31.561299
cloudflare-bypass | [19:09:35] INFO - cf_bypasser.server.routes - Successfully generated HTML content (97239 chars) and 1 cookies in 21336ms
cloudflare-bypass | [19:09:35] INFO - cf_bypasser.server.routes - Cloudflare cookies: ['cf_clearance']
cloudflare-bypass | INFO: 172.16.1.1:55896 - "GET /html?url=https://TARGET_WEBSITE_3 HTTP/1.1" 200 OK
Can you try the new API?
Like that;
curl "http://localhost:8000/" -H "x-hostname: TARGET_WEBSITE_3.com"
If you're having problems with the new version, use the legacy image until I develop a solution for it.
Can you try the new API?
It happens with the new API, but much less. If I run 12 concurrent curl jobs with parallel I get 1 NoneType error.
I also get these new errors:
cloudflare-bypass | [20:11:23] INFO - root - Error in get_or_generate_cookies: BrowserContext.new_page: Target page, context or browser has been closed
and
cloudflare-bypass | [20:11:40] INFO - root - Error solving Cloudflare challenge: Page.goto: Target page, context or browser has been closed
Not sure if these are related, and calling 12 jobs is more than my real world use, so it might not be truly representative, but just putting it here in case it helps.
It appears the issue was with Camouflox itself. Initializing a Camouflox browser isn’t thread-safe, particularly due to their browserforge dependency. It’s working now.
# Use global lock to serialize browser initialization (browserforge is not thread-safe)
async with get_browser_init_lock():
browser = await AsyncCamoufox(
headless=True,
geoip=True if proxy else False,
humanize=False,
os=selected_os,
locale=lang if lang else "en-US",
i_know_what_im_doing=True,
config={'forceScopeAccess': True, **random_config},
disable_coop=True,
main_world_eval=True,
addons=[os.path.abspath(ADDON_PATH)],
block_images=False,
block_webrtc=True,
enable_cache=False,
).__aenter__()
https://github.com/sarperavci/CloudflareBypassForScraping/actions/runs/19488313161/job/55775178951