Run parallel browsers from one node process

[ClassAxion]

Does this package support for run multiple browsers at once? Like 10-50 parallel browsers from the same node process?

I'm trying but browsers starting very very long or even timeout at 300s and freezes.

How to do this correctly? I'm using plugin.launch with headless: 'new' and useFingerprint for launching browsers

[CheshireCaat]

At this point in time, the launch of browsers is synchronized, but after that you can work with them in parallel in any way.

Parallel unsynchronized launch isn't yet supported due to some limitations - we may fix this in the future, so we'll leave the ticket open for now.

Launching the browser takes longer than default one, as it requires additional configuration and the presence of an engine that is downloaded once. If a timeout error occurs, you can change the default timeout to any value you like via the FINGERPRINT_TIMEOUT environment variable.

If you're having trouble launching multiple browsers and then working with them, or if I misunderstood you, please describe the problem in more detail and attach sample code.

[ClassAxion]

So I should use some mutex for plugin.launch option and launch only one browser at a time? Then I can use page.goto etc. without any mutex, sync. etc?

As far as I understood correctly, the problem is that the browsers uses the same data directory and this causes a problem with unsynchronized launch of multiple browsers?

[ClassAxion]

I just tested my code which didn't work with async-mutex putted on plugin.launch and it works fine. I think that this is the current workaround

[CheshireCaat]

I still don't understand what your problem is and why you need to use additional mutex inside launch. Here's an example straight from the repository demonstrating running multiple browsers:

require('dotenv').config();
const { plugin } = require('puppeteer-with-fingerprints');

async function main(index) {
  const browser = await plugin.launch();

  const page = await browser.newPage();
  await page.goto(`https://jsonplaceholder.typicode.com/todos/${index + 1}`, { waitUntil: 'domcontentloaded' });

  const todo = JSON.parse(await page.$eval('pre', (pre) => pre.innerText));

  await browser.close();
  return todo;
}

Promise.all([0, 1, 2].map(main)).then((todos) => {
  console.log({ todos });
});

I tested it many times and it worked. Try running it and let me know what specific problems you're having with it.

[ClassAxion]

but this example doesn't use useFingerprint method, if you use useFingerprint then launch should freeze after some time (for example if you have 20+ workers that runs unsynchronised and restart the browser every 5 minutes)

[bablosoft]

@ClassAxion can you attach minimum runnable source code and what error do you get.

Please avoid using FingerprintSwitcher key in your code.

[emretulek]

Hello, I'm having the same problem.

project_folder -> index.js

export const index = async (plugin) => {
    const fingerprint = await plugin.fetch('key', {
        tags: ['Microsoft Windows', 'Chrome'],
    });
    log.yellow("STEP: fingerprints fetched");

    plugin.useProxy(proxy, {
        changeGeolocation: true,
    });

    log.white("STEP: proxy setted");

    plugin.useFingerprint(fingerprint);

    log.white("STEP: figerprint using");

    const browser = await plugin.launch({
        headless: isHeadless,
        // minBrowserVersion: 102,
        // minWidth: 1024,
        // minHeight: 768,
        // maxWidth: 1920,
        // maxHeight: 1080,
    });

    log.white("STEP: browser launched");
}

project_folder -> sub_folder1

import { plugin } from "puppeteer-with-fingerprints";
import {index} from "../index.js";

await index(plugin);

It is installed in the puppeteer-with-fingerprints subdirectory, so each browser is used from a different directory. After a few hours the lunch method freezes. ctr + c doesn't work, console hangs. If I close other processes as well, the frozen console exits. Mostly the error does not appear.

At the same place, I sometimes encounter the following error.

node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}

Node.js v19.0.0

[CheshireCaat]

@emretulek I don't see any issues here, your code works fine for me - please, describe your problem in more details. Also keep in mind that you must close browsers properly whether you are using a plugin or not. Otherwise, it can lead to other problems, such as excessive consumption of resources, and the process can remain open, including because of this.

[emretulek]

@CheshireCaat When I open multiple browsers, the most noticeable problem is that processes freeze in relation to one another. For example, if 5 processes are started at the same time, and the 2nd process is frozen, it cannot be terminated even with ctr+c. But it freezes in conjunction with one of the processes 1 - 3 - 4 - 5. When 1-4-5 is closed, nothing changes, while 3 is turned off, 2 continues from where it left off. (It doesn't have to be 3, it can be any one specifically.)