Playwright tests not working with GitHub Action? browserType.launch: Executable doesn't exist at /github/home/.cache/ms-playwright/chromium-1076/chrome-linux/chrome

[GeorgeXCV]

Version info:

v2.0.0-37

Running this command:

run tests/my-test.yml --output ./report.json

I expected to see this happen:

The test to run like it does locally.

Instead, this happened:

worker error, id: 1 browserType.launch: Executable doesn't exist at /github/home/.cache/ms-playwright/chromium-1076/chrome-linux/chrome
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝
    at Array.scenario (/home/node/artillery/node_modules/artillery-engine-playwright/index.js:56:38)
    at runScenario (/home/node/artillery/node_modules/@artilleryio/int-core/lib/runner.js:331:32)
    at EventEmitter.<anonymous> (/home/node/artillery/node_modules/@artilleryio/int-core/lib/runner.js:203:25)
    at EventEmitter.emit (/home/node/artillery/node_modules/eventemitter3/index.js:181:35)
    at UniformProcess.<anonymous> (/home/node/artillery/node_modules/@artilleryio/int-core/lib/phases.js:220:[12](https://github.com/nansen-ai/nansen-artillery/actions/runs/6467513570/job/17557698774#step:9:13))
    at /home/node/artillery/node_modules/arrivals/lib/index.js:47:10
    at NanoTimer.setInterval (/home/node/artillery/node_modules/nanotimer/lib/nanotimer.js:2[20](https://github.com//actions/runs/6467513570/job/17557698774#step:9:21):5)
    at Immediate._onImmediate (/home/node/artillery/node_modules/nanotimer/lib/nanotimer.js:203:68) ***
  name: 'Error'

Files being used: Workflow:

name: Load Test

on:
    workflow_dispatch:
 
jobs:
  artillery:
    runs-on: ubuntu-latest
 
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 18

      - name: Setup bun dependencies
        uses: oven-sh/setup-bun@v1
        with:
          bun-version: latest
        
      - name: Install dependencies
        run: |
          bun install
          npx playwright install

      - name: Write the serviceAccountKey.json file 📝
        run: |
            echo '${{ secrets.SERVICE_ACCOUNT_KEY }}' > serviceAccountKey.json
 
      - name: Execute load tests
        uses: artilleryio/action-cli@v1
        with:
          command: run tests/my-test.yml --output ./report.json
        
      - name: Upload artifact
        uses: actions/upload-artifact@v3
        if: always()
        with:
          name: artillery-report
          # Reference the generated report in the file system.
          path: ./report.json

[hassy]

Try changing npx playwright install to npx [email protected] install. Artillery's Playwright integration uses Playwright v1.37.1 (source), and browsers in Playwright need to be installed for specific versions.

[hassy]

Or better yet: run your tests on AWS Fargate (docs). That will let you generate more load (GitHub Actions is not a great place to actually execute the tests from), and you won't need to install browsers explicitly in the workflow.

[GeorgeXCV]

npx [email protected] install

Still get the same error.

[TestItEver]

Or better yet: run your tests on AWS Fargate (docs). That will let you generate more load (GitHub Actions is not a great place to actually execute the tests from), and you won't need to install browsers explicitly in the workflow.

Hi @hassy! I've just got the same error on AWS Fargate, although a couple of days ago it works correctly. I didn't update or change anything in my project so far:

worker error, id: 1 browserType.launch: Executable doesn't exist at /ms-playwright/chromium-1076/chrome-linux/chrome

VERSION INFO: Artillery: 2.0.0-37 Node.js: v18.17.0 OS: win32

[harv-singh]

➕ was able to reproduce this

[nlyn]

I'm also getting this on Fargate

[freyjadomville]

We are also able to repro this with our Playwright test using Fargate, on the latest available release (v2.0.0-38).

[freyjadomville]

I think I have a way to get this working inside the flow itself so that the initialisation is done once per worker on Fargate - it behaves locally, so I'm hoping that I can execute the same command set on Fargate, depending on if npx itself is executable inside the Fargate cluster instance. I have some local issues to fix in my current Playwright test, but I'll report back separately once I have something working or not:

before:
  flow:
    - function: "playwrightInit"

async function playwrightInit(_context, _event, next) {
  const exec = require('child_process').exec;
  const util = require('util');

  const execPromise = util.promisify(exec)
  const { stdout, stderr } = await execPromise('npx [email protected] install');
  console.log('stdout: ', stdout);
  console.log('stderr: ', stderr);
  next();
}

EDIT: I suspect this isn't quite what I want in the multi-user case, so I may edit this comment to allow for it to be worker safe

EDIT 2: Reading the docs further, it appears this is referencing an "Artillery worker", so ignore my previous edit, I was confusing this for Web Workers in JS.

[freyjadomville]

Unfortunately, this didn't work on Fargate, there's no stdout or stderr to check (unlike when I'm running things locally with this step, which does work), and I therefore believe the npm-cli packages aren't in the image that runs Artillery. A shame, but as a result we're all blocked on this until the platform-fargate package can initialise the container with playwright browsers included as a fresh image.

[tuukkasp]

I'm getting the same issue on Fargate using

"artillery": "^2.0.0-38",
"@playwright/test": "^1.39.0",

[freyjadomville]

OK, I've been able to replicate the issue inside a Distrobox of Debian 12 (created via distrobox create --image debian:stable-backports -n Debian) and on Arch Linux locally when I don't have a copy of the playwright browsers installed.

In short, regardless of the browser choice or playwright version, npx [email protected] install will always install the latest version of the browsers, and for some reason Artillery is always depending on /ms-playwright/chromium-1076/chrome-linux/chrome.

This would explain why we didn't see it before locally and why things are failing during image construction.

Pinging @hassy again for info and good measure - you need to fix this on your side, as we can't vendor this type of thing as a workaround in a library - it's dependent on the obfuscated parts of your artillery-pro offering. Artillery as an organisation should be keeping on top of this type of thing and making sure these types of things work reliably, even for new installations.

EDIT: I potentially stand corrected, it is viable for us to depend on both artillery and Playwright explicitly in our project to get local builds working again, and artillery-pro would only affect Fargate

[hassy]

Going to add more context here for anyone running into similar issue and trying to debug. (Information is accurate at the time of writing and will probably go out of date soon)

Our Playwright integration is currently pinned to v1.39.0 of Playwright. (We'd like to offer the ability to pick any version of Playwright, but as the Playwright team do a good job of maintaining backwards-compatibility, it's not a priority right now.)

If you're running a test on Fargate (with run-fargate) with the most recent version of Artillery, the Docker image that will be used to execute the tests already includes Playwright v1.39.0 and an installation of Chromium. It should all just work without you needing to do anything. If it doesn't, then it's a bug we need to fix.

If you're running tests elsewhere (e.g. locally, or in GitHub Actions), then you have to make sure to install the browsers. The path where the browsers are expected to be found (e.g. /ms-playwright/chromium-1076/chrome-linux/chrome) is set by Playwright itself and not by Artillery, and it changes between version of Playwright, so you have to make sure you're installing v1.39.0 exactly.

[freyjadomville]

That makes more sense - we were using artillery in the package.json and likely having version conflicts in package-lock.json because of it. Now that I am aware that'll cause non-ideal results, I'll switch our codebase to an npx artillery approach for Artillery itself. Thanks for the clarification.

[TestItEver]

If you're running a test on Fargate (with run-fargate) with the most recent version of Artillery, the Docker image that will be used to execute the tests already includes Playwright v1.39.0 and an installation of Chromium. It should all just work without you needing to do anything. If it doesn't, then it's a bug we need to fix.

Hi @hassy! That is exactly, what we are doing: running a test on Farget with npx artillery run-fargate and get the error " Executable doesn't exist at /ms-playwright/chromium-1076/chrome-linux/chrome". The artillery version is 2.0.0.-37. I will test it with the most recent version.

[lazzarello]

Going to add more context here for anyone running into similar issue and trying to debug. (Information is accurate at the time of writing and will probably go out of date soon)

Our Playwright integration is currently pinned to v1.39.0 of Playwright. (We'd like to offer the ability to pick any version of Playwright, but as the Playwright team do a good job of maintaining backwards-compatibility, it's not a priority right now.)

If you're running a test on Fargate (with run-fargate) with the most recent version of Artillery, the Docker image that will be used to execute the tests already includes Playwright v1.39.0 and an installation of Chromium. It should all just work without you needing to do anything. If it doesn't, then it's a bug we need to fix.

If you're running tests elsewhere (e.g. locally, or in GitHub Actions), then you have to make sure to install the browsers. The path where the browsers are expected to be found (e.g. /ms-playwright/chromium-1076/chrome-linux/chrome) is set by Playwright itself and not by Artillery, and it changes between version of Playwright, so you have to make sure you're installing v1.39.0 exactly.

~~This appears to have changed again. I'm new to Playwright and Artillery so it's my first encounter with the issue. It seems like many have worked around it. I'll spend some time trying to do the same.~~

Never mind, I had a more current version of the browser deps in the cache and hadn't yet installed the older ones. Indeed it works with the version of Playwright specified in packages.json

lee@suzanne:~/src/artillery_tutorial$ npm install [email protected]

added 2 packages in 4s
lee@suzanne:~/src/artillery_tutorial$ artillery run playwright.yml 
Test run id: tth9d_6eebc96bz85p8kfrghmfjk78m8afg_rqyd
Phase started: unnamed (index: 0, duration: 1s) 15:33:03(-0800)

Phase completed: unnamed (index: 0, duration: 1s) 15:33:04(-0800)

worker error, id: 1 browserType.launch: Executable doesn't exist at /home/lee/.cache/ms-playwright/chromium-1084/chrome-linux/chrome
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝
    at Array.scenario (/usr/lib/node_modules/artillery/node_modules/artillery-engine-playwright/index.js:67:38)
    at runScenario (/usr/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/runner.js:373:32)
    at EventEmitter.<anonymous> (/usr/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/runner.js:203:25)
    at EventEmitter.emit (/usr/lib/node_modules/artillery/node_modules/eventemitter3/index.js:181:35)
    at UniformProcess.<anonymous> (/usr/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/phases.js:220:12)
    at /usr/lib/node_modules/artillery/node_modules/arrivals/lib/index.js:47:10
    at NanoTimer.setInterval (/usr/lib/node_modules/artillery/node_modules/nanotimer/lib/nanotimer.js:220:5)
    at Immediate._onImmediate (/usr/lib/node_modules/artillery/node_modules/nanotimer/lib/nanotimer.js:203:68) {
  name: 'Error'
}
lee@suzanne:~/src/artillery_tutorial$ ls -ld /home/lee/.cache/ms-playwright/chromium-*
drwxrwxr-x 3 lee lee 4096 Dec 11 11:42 /home/lee/.cache/ms-playwright/chromium-1091
lee@suzanne:~/src/artillery_tutorial$ npx playwright --version
Version 1.39.0

[hassy]

@lazzarello glad it's working now!

[BhavnaSoni1991]

Hi @hassy! I am still getting error "worker error, id: 1 browserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium-1084/chrome-linux/chrome" in pipeline even when I am installing playwright version 1.39.0. Below is my Dockerfile.

FROM artilleryio/artillery:2.0.4

RUN npm install -g [email protected] [email protected] [email protected] &&
npm cache clean --force &&
rm -rf /root/.cache &&
rm -rf /ms-playwright/firefox* &&
rm -rf /ms-playwright/webkit*

ENTRYPOINT ["/home/node/artillery/bin/run"]