pyppeteer.errors.NetworkError: Request is already handled

[skullfrac]

Expected Behavior

demo.py, after adjusting the proxy settings, connecting to the recaptcha demo site and returning a result.

Current Behavior

it starts up normally, as soon as communication with the server is expected it crashes with following error:

 Exception in callback CompatEventEmitter._emit_run.<locals>._callback(<Task finishe...dy handled.')>) at /usr/local/lib/python3.7/dist-packages/pyee/_compat.py:58
handle: <Handle CompatEventEmitter._emit_run.<locals>._callback(<Task finishe...dy handled.')>) at /usr/local/lib/python3.7/dist-packages/pyee/_compat.py:58>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.7/dist-packages/pyee/_compat.py", line 62, in _callback
    self.emit('error', exc)
  File "/usr/local/lib/python3.7/dist-packages/pyee/_base.py", line 106, in emit
    self._emit_handle_potential_error(event, args[0] if args else None)
  File "/usr/local/lib/python3.7/dist-packages/pyee/_base.py", line 83, in _emit_handle_potential_error
    raise error
  File "/usr/local/lib/python3.7/dist-packages/nonocaptcha/solver.py", line 110, in handle_request
    await request.continue_()
  File "/usr/local/lib/python3.7/dist-packages/pyppeteer/network_manager.py", line 483, in continue_
    raise NetworkError('Request is already handled.')

Steps to reproduce

1. Load this adjusted Dockerfile

# Build the Docker container by running "docker build ." in the same folder as
# Dockerfile. After the build is complete, run "docker images" and copy the
# most recent create image. Then run the command "docker run -i -t COPIEDIMAGE"
# which will place you in the shell of the newly created container.
# All files are located in /nonocaptcha.


# This Dockerfile assumes all required files/folders are in the relative
# folder:
# - nonocaptcha.yaml
# - examples/app.py
# - pocketsphinx (folder)
# You may want to add proxies.txt at the bottom of this file.

# We are using Ubuntu 16.04 for the base Docker image
FROM ubuntu:16.04

# This installs all the required packages for Python3.6, Chrome, and
# Pocketsphinx
RUN apt-get update \
    && apt-get install -y \
    libpangocairo-1.0-0 \
    libx11-xcb1 \
    libxcomposite1 \
    libxcursor1 \
    libxdamage1 \
    libxi6 \
    libxtst6 \
    libnss3 \
    libcups2 \
    libxss1 \
    libxrandr2 \
    libgconf-2-4 \
    libasound2 \
    libasound2-dev \
    libatk1.0-0 \
    libgtk-3-0 \
    gconf-service \
    libappindicator1 \
    libc6 \
    libcairo2 \
    libcups2 \
    libdbus-1-3 \
    libexpat1 \
    libfontconfig1 \
    libgcc1 \
    libgdk-pixbuf2.0-0 \
    libglib2.0-0 \
    libnspr4 \
    libpango-1.0-0 \
    libpulse-dev \
    libstdc++6 \
    libx11-6 \
    libxcb1 \
    libxext6 \
    libxfixes3 \
    libxrender1 \
    libxtst6 \
    ca-certificates \
    fonts-liberation \
    lsb-release \
    xdg-utils \
    build-essential \
    ffmpeg \
    swig \
    software-properties-common curl \
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get remove -y software-properties-common \
    && apt autoremove -y \
    && apt-get update \
    && apt-get install -y python3.7 \
    python3.7-dev \
    && curl -o /tmp/get-pip.py "https://bootstrap.pypa.io/get-pip.py" \
    && python3.7 /tmp/get-pip.py \
 && apt-get remove -y curl \
    && apt autoremove -y \
    && pip install nonocaptcha \
    && pip install pyppeteer

RUN python3.7 -c 'import pyppeteer; pyppeteer.chromium_downloader.download_chromium()'

# Copies required files for running nonoCAPTCHA to the Docker container.
# You can comment out pocketsphinx if you aren't using Pocketsphinx.
RUN mkdir /nonocaptcha
ADD pocketsphinx /nonocaptcha/pocketsphinx
ADD nonocaptcha.yaml /nonocaptcha
# ADD proxies.txt /nonocaptcha/proxies.txt

# This determines which file you want to copy over to the Docker container,
# by default the aiohttp server is copied to the container.
ADD examples/* /nonocaptcha/

# Uncomment the lines below if you want to autostart the app and expose the
# port on your machine, which can be accessed by going to http://localhost:5000
# RUN python3.6 /nonocaptcha/app.py
# EXPOSE 5000

2. clone pocketspinx into docker dir
3. cp nonocaptcha/examples into docker dir
4. cp nonocaptcha.yml into docker dir
5. build and run
6. adjust /nonocaptcha/demo.py settings
7. in container: cd /nonocaptcha && python3.7 demo.py

Additional information

i updated to python 3.7 as expected an error in aiohttp to cause the trouble first, i also added the additional command to install chrome successfully. taken from another Issue here. The setup is headless.

[MacKey-255]

You need update the dependencies

[singgi11]

well, I'm having the same error as ts - it just opens the chromium and then it ended with pyppeteer.errors.NetworkError: Request is already handled.

[Francesco149]

this happens because there's two request listeners at the same time. merge the block_images listener with the inject_widget one to fix.

[MacKey-255]

well, I'm having the same error as ts - it just opens the chromium and then it ended with pyppeteer.errors.NetworkError: Request is already handled.

Look my fork here https://github.com/MacKey-255/GoodByeCatpcha

[weaming]

Same issue

2020-05-09 15:55:27,559 Exception in callback <function CompatEventEmitter._emit_run.<locals>._callback at 0x7f78198f1158>
handle: <Handle CompatEventEmitter._emit_run.<locals>._callback>
Traceback (most recent call last):
  File "uvloop/cbhandles.pyx", line 70, in uvloop.loop.Handle._run
  File "/usr/local/lib/python3.6/dist-packages/pyee/_compat.py", line 63, in _callback
    self.emit('error', exc)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_base.py", line 111, in emit
    self._emit_handle_potential_error(event, args[0] if args else None)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_base.py", line 83, in _emit_handle_potential_error
    raise error
  File "/usr/local/lib/python3.6/dist-packages/nonocaptcha/solver.py", line 110, in handle_request
    await request.continue_()
  File "/usr/local/lib/python3.6/dist-packages/pyppeteer/network_manager.py", line 483, in continue_
    raise NetworkError('Request is already handled.')
pyppeteer.errors.NetworkError: Request is already handled.

How to fix it ? Any one has an PR for this?

[Francesco149]

@weaming https://github.com/Francesco149/nonoCAPTCHA/commit/168c834cbd2612ceb2841a88a56822caf7adba79

[weaming]

@weaming Francesco149@168c834

@Francesco149 I'm using your fork repo to test, but an error raised:

root@eb0004fc9a4e:/app2/nonoCAPTCHA/examples# python3.6 demo.py
/usr/local/lib/python3.6/dist-packages/nonocaptcha_fork/base.py:20: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  settings = yaml.load(f)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/nonocaptcha_fork/solver.py", line 64, in start
    self.browser = await self.get_new_browser()
  File "/usr/local/lib/python3.6/dist-packages/nonocaptcha_fork/solver.py", line 179, in get_new_browser
    browser = await self.launcher.launch()
  File "/usr/local/lib/python3.6/dist-packages/pyppeteer_fork/launcher.py", line 174, in launch
    self.browserWSEndpoint = self._get_ws_endpoint()
  File "/usr/local/lib/python3.6/dist-packages/pyppeteer_fork/launcher.py", line 211, in _get_ws_endpoint
    with urlopen(url) as f:
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.6/urllib/request.py", line 1321, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1354, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 276, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

2020-05-10 05:04:41,638 0 Remote end closed connection without response <class 'http.client.RemoteDisconnected'>
2020-05-10 05:04:41,670 0 Browser closed
2020-05-10 05:04:41,671 0 Time elapsed: 1.2698609828948975

As I'm using it in a docker container, I set the headless to true in nonocaptcha.yaml.

Do you have any suggestion?