[Bug]: Socks5 Proxy get error net::ERR_SOCKS_CONNECTION_FAILED

[zmzimpl]

Minimal, reproducible example

const puppeteer = require("puppeteer");
const { execSync, spawn } = require("child_process");

const BASE_PORT = 9222;
const TOTAL_WINDOWS = 1;
const CHROME_PATH =
  "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe";
const BASE_PROXIES_PORT = 30000;

const CACHE_PATH = "D:\\chromes\\users\\";

const proxyUser = "xxx";
const proxyPass = "xxx";

const userAgents = [
  "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36",
];

async function automateBrowser(port, userAgent) {
  console.log("port", port);
  console.log("userAgent", userAgent);
  const browserURL = `http://127.0.0.1:${port}`;
  const browser = await puppeteer.connect({ browserURL });

  const pages = await browser.pages();
  const page = pages.length ? pages[0] : await browser.newPage();

  await page.authenticate({
    username: proxyUser,
    password: proxyPass,
  });
    await page.setUserAgent(userAgent);

    await page.goto("https://ip.me");


  // await browser.close();
}

(async () => {
  for (let i = 0; i < TOTAL_WINDOWS; i++) {
    const port = BASE_PORT + i;
    const userAgent = userAgents[i % userAgents.length];
    const userDataDir = `${CACHE_PATH}\\${i}`;
    const proxy = `socks5://156.254.207.151:44385`;

    const chromeInstance = spawn(CHROME_PATH, [
      `--remote-debugging-port=${port}`,
      `--proxy-server=${proxy}`,
      `--host-resolver-rules="MAP * ~NOTFOUND , EXCLUDE 156.254.207.151"`,
      `--user-data-dir=${userDataDir}`,
    ]);

    console.log(chromeInstance.pid);
    await new Promise((resolve) => setTimeout(resolve, 3000));

    await automateBrowser(port, userAgent);

    // execSync(`taskkill /PID ${chromeInstance.pid} /F`);
  }
})();

Error string

Error: net::ERR_SOCKS_CONNECTION_FAILED

Bug behavior

• [ ] Flaky
• [ ] PDF

Background

I'm trying to proxy with socks5 according https://www.chromium.org/developers/design-documents/network-stack/socks-proxy/ and I'm getting an error Error: net::ERR_SOCKS_CONNECTION_FAILED at https://ip.me

Expectation

would like to be able to access the web page properly

Reality

getting an error Error: net::ERR_SOCKS_CONNECTION_FAILED

Puppeteer configuration file (if used)

No response

Puppeteer version

21.1.0

Node version

v18.14.0

Package manager

npm

Package manager version

9.3.1

Operating system

Windows

[github-actions[bot]]

This issue was not reproducible. Please check that your example runs locally and the following:

• Ensure the script does not rely on dependencies outside of puppeteer and puppeteer-core.
• Ensure the error string is just the error message.

  • Bad:

    Error: something went wrong
      at Object.<anonymous> (/Users/username/repository/script.js:2:1)
      at Module._compile (node:internal/modules/cjs/loader:1159:14)
      at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
      at Module.load (node:internal/modules/cjs/loader:1037:32)
      at Module._load (node:internal/modules/cjs/loader:878:12)
      at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
      at node:internal/main/run_main_module:23:47
    

  • Good: Error: something went wrong.

• Ensure your configuration file (if applicable) is valid.
• If the issue is flaky (does not reproduce all the time), make sure 'Flaky' is checked.
• If the issue is not expected to error, make sure to write 'no error'.

Once the above checks are satisfied, please edit your issue with the changes and we will try to reproduce the bug again.

---

Analyzer run

[jrandolf-2]

Looks to me like the proxy is problematic. Have you verified that you can navigate to ip.me through the proxy? (Without using Chrome or Puppeteer)

[zmzimpl]

Looks to me like the proxy is problematic. Have you verified that you can navigate to ip.me through the proxy? (Without using Chrome or Puppeteer)

Yes, I verified the proxy, and it works fine with socks-proxy-agent. Test Code:

const https = require('https');
const { SocksProxyAgent } = require('socks-proxy-agent');

const agent = new SocksProxyAgent(
	'socks://username:[email protected]:33544'
);

https.get('https://ip.me', { agent }, (res) => {
	console.log(res.statusCode);
});

Output: 200

[zmzimpl]

Looks to me like the proxy is problematic. Have you verified that you can navigate to ip.me through the proxy? (Without using Chrome or Puppeteer)

Looks to me like the proxy is problematic. Have you verified that you can navigate to ip.me through the proxy? (Without using Chrome or Puppeteer)

Yes, I verified the proxy, and it works fine with socks-proxy-agent. Test Code:

const https = require('https');
const { SocksProxyAgent } = require('socks-proxy-agent');

const agent = new SocksProxyAgent(
	'socks://username:[email protected]:33544'
);

https.get('https://ip.me', { agent }, (res) => {
	console.log(res.statusCode);
});

Output: 200

Additional: I use setRequestInterceptor(true) and create an agent by the proxy, it work fine too

  await page.setRequestInterception(true);
  page.on("request", async (request) => {
    if (
      request.url().startsWith("data") ||
      request.url().startsWith("blob") ||
      request.url().startsWith("chrome-extension") ||
      request.url().startsWith("chrome-devtools") ||
      request.url().endsWith(".png") ||
      request.url().endsWith(".jpg") ||
      request.url().endsWith(".jpeg") ||
      request.url().endsWith(".gif") ||
      request.url().endsWith(".css") ||
      request.url().endsWith(".woff") ||
      request.url().endsWith(".woff2") ||
      request.url().endsWith(".ttf") ||
      request.url().endsWith(".svg") ||
      request.url().endsWith(".ico") ||
      request.url().endsWith(".mp4") ||
      request.url().endsWith(".mp3") ||
      request.url().endsWith(".avi") ||
      request.url().endsWith(".mov") ||
      request.url().endsWith(".flv") ||
      request.url().endsWith(".wmv") ||
      request.url().endsWith(".rmvb") ||
      request.url().endsWith(".mkv") ||
      request.url().endsWith(".rm") ||
      request.url().endsWith(".asf") ||
      request.url().endsWith(".ram") ||
      request.url().endsWith(".divx") ||
      request.url().endsWith(".mpe") ||
      request.url().endsWith(".mpeg") ||
      request.url().endsWith(".vob") ||
      request.url().endsWith(".dat") ||
      request.url().endsWith(".f4v") ||
      request.url().endsWith(".m4v") ||
      request.url().endsWith(".ts") ||
      request.url().endsWith(".3gp") ||
      request.url().endsWith(".3g2") ||
      request.url().endsWith(".mk3d") ||
      request.url().endsWith(".ogv") ||
      request.url().endsWith(".webm") ||
      request.url().endsWith(".m3u8") ||
      request.url().endsWith(".m3u")
    ) {
      request.continue();
    } else {
      try {
        const agent = new SocksProxyAgent(
          "socks://username:[email protected]:33544"
        );
        console.log("request.url()", request.url());
        const response = await axios.request({
          url: request.url(),
          httpAgent: agent,
          httpsAgent: agent,
          proxy: false,
          data: request.postData(),
          headers: request.headers(),
          method: request.method(),
        });
        // 4. Return response to Chrome
        await request.respond({
          body: response.data,
          headers: response.headers,
          status: response.status,
        });
      } catch (error) {
        console.log("error", error.message);
      }
    }

[stale[bot]]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[qiuyeduxin]

hello，Is this problem solved? I also encountered it.

[zmzimpl]

hello，Is this problem solved? I also encountered it.

Not really, I solved the problem by using an HTTP proxy locally.

[zmzimpl]

I solved this problem in this way:

import * as http from 'http';
import * as url from 'url';
import {SocksClient} from 'socks';
import {EventEmitter} from 'events';
import {SocksProxyAgent} from 'socks-proxy-agent';
import type * as internal from 'stream';
import type {SocksCommandOption} from 'socks/typings/common/constants';
import {createLogger} from '../../../shared/utils/logger';
import {PROXY_LOGGER_LABEL} from '../constants';

const logger = createLogger(PROXY_LOGGER_LABEL);

interface SocketOptions {
  listenHost: string;
  listenPort: number;
  socksHost: string;
  socksPort: number;
  socksUsername?: string;
  socksPassword?: string;
}

interface SocksProxyOptions {
  ipaddress: string;
  port: number;
  type: 5 | 4;
  userId?: string;
  password?: string;
}

class HttpProxy extends EventEmitter {
  opt: SocketOptions = {
    listenHost: 'localhost',
    listenPort: 12333,
    socksHost: 'localhost',
    socksPort: 1080,
  };
  proxy: SocksProxyOptions = {
    ipaddress: '127.0.0.1',
    port: 7890,
    type: 5,
  };
  constructor(options: SocketOptions) {
    super();
    this.opt = {
      ...this.opt,
      ...options,
    };
    this.proxy = {
      ipaddress: this.opt.socksHost,
      port: this.opt.socksPort,
      type: 5,
      userId: this.opt.socksUsername || '',
      password: this.opt.socksPassword || '',
    };
  }

  _request(
    proxy: SocksProxyOptions,
    uReq: http.IncomingMessage,
    uRes: http.ServerResponse<http.IncomingMessage> & {
      req: http.IncomingMessage;
    },
  ) {
    const u = url.parse(uReq.url!);
    const socksAgent = new SocksProxyAgent(
      `socks://${proxy.userId}:${proxy.password}@${proxy.ipaddress}:${proxy.port}`,
    );

    const options = {
      hostname: u.hostname,
      port: u.port || 80,
      path: u.path,
      method: uReq.method || 'get',
      headers: uReq.headers,
      agent: socksAgent,
    };
    const pReq = http.request(options);
    pReq
      .on('response', pRes => {
        pRes.pipe(uRes);
        uRes.writeHead(pRes.statusCode!, pRes.headers);
        this.emit('request:success');
      })
      .on('error', e => {
        uRes.writeHead(500);
        uRes.end('Connection error\n');
        this.emit('request:error', e);
      });
    uReq.pipe(pReq);
  }

  _connect(
    proxy: SocksProxyOptions,
    uReq: http.IncomingMessage,
    uSocket: internal.Duplex,
    uHead: Buffer,
  ) {
    const u = url.parse(`http://${uReq.url}`);
    const options = {
      proxy,
      destination: {host: u.hostname!, port: u.port ? +u.port! : 80},
      command: 'connect' as SocksCommandOption,
    };
    SocksClient.createConnection(options, (error, pSocket) => {
      if (error) {
        uSocket?.write(`HTTP/${uReq.httpVersion} 500 Connection error\r\n\r\n`);
        this.emit('connect:error', error);
        return;
      }
      pSocket?.socket.pipe(uSocket);
      if (pSocket?.socket) {
        uSocket?.pipe(pSocket?.socket);
      }
      pSocket?.socket.on('error', err => {
        this.emit('connect:error', err);
      });
      uSocket.on('error', err => {
        this.emit('connect:error', err);
      });
      pSocket?.socket.write(uHead);
      uSocket?.write(`HTTP/${uReq.httpVersion} 200 Connection established\r\n\r\n`);
      this.emit('connect:success');
      pSocket?.socket.resume();
    });
  }

  start() {
    const server = http.createServer();
    server.on('connect', (...args) => this._connect(this.proxy, ...args));
    server.on('request', (...args) => this._request(this.proxy, ...args));
    return server.listen(this.opt.listenPort, this.opt.listenHost);
  }
}

export default function SocksServer(opt: SocketOptions) {
  logger.info(
    `Listen on ${opt.listenHost}:${opt.listenPort}, and forward traffic to ${opt.socksHost}:${opt.socksPort}`,
  );
  const proxy = new HttpProxy(opt);
  return proxy.start();
}

and use it:

  const proxyServer = SocksServer({
    listenHost,
    listenPort,
    socksHost,
    socksPort: +socksPort,
    socksUsername,
    socksPassword,
  });

  proxyServer.on('connect:error', err => {
    logger.error(err);
  });
  proxyServer.on('request:error', err => {
    logger.error(err);
  });

[dimaslanjaka]

I also just found this, but no one proxy working. but when I use the proxy in cURL, the proxy working.

const puppeteer = require('puppeteer');
const proxy = require('puppeteer-page-proxy');

(async () => {
  // Replace 'socks5://your-socks-proxy:port' with the actual SOCKS5 proxy address and port
  const socksProxy = 'socks5://your-socks-proxy:port';

  // Launch the browser
  const browser = await puppeteer.launch();

  // Create a new page
  const page = await browser.newPage();

  // Enable the proxy on the page
  await proxy(page, socksProxy);

  // Navigate to a website
  await page.goto('https://example.com');

  // Rest of your Puppeteer script...

  // Close the browser
  await browser.close();
})();

[github-actions[bot]]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[github-actions[bot]]

We are closing this issue. If the issue still persists in the latest version of Puppeteer, please reopen the issue and update the description. We will try our best to accommodate it!