Cypress Github action pauses for 3-5min in the middle of tests that only take 30seconds to run, logging ContextResult::kFatalFailure: fail_if_major_perf_caveat + swiftshader

[jfarris587]

Using: "cypress": "^9.0.0",

Cypress on MacOS local machine runs within 2min. Each test takes 30-45seconds and there are no long pauses between tests.

Using the Github action, running parallelization with 3 machines, each machine will pause and wait 1-3min in the middle of a test. Check out the logs below, you see there is a 3min gap between logs where nothing happens.

Coincidentally, its the same time ContextResult::kFatalFailure: fail_if_major_perf_caveat + swiftshader is logged. Notice the delay from 22:20 to 22:23 where nothing happens

This is repeatable and happens on ever single run in github actions. In addition, potentially related, but maybe not, the dashboard recording never shows the full test video and always ends early. It seems like it may be a CPU/RAM issue however, its a basic NextJS app with some basic cypress tests. Can NextJS just not run in Github actions with cypress?

https://user-images.githubusercontent.com/31263423/144269661-38d21492-f1c3-4ab7-a03a-d94f7bbf148d.mp4

The workflow file is as follows:

# This workflow does not have access to secrets and a read-write token when called by dependabot
name: CI
on:
  pull_request:
    branches: [development]

jobs:
  yarn:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: cache node modules
        uses: actions/cache@v2
        with:
          path: node_modules
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}

      - name: Setup Node.js environment
        uses: actions/[email protected]
        with:
          node-version: 14.15.4

      - name: yarn install
        run: yarn install --frozen-lockfile

  build:
    needs: yarn
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: cache node modules
        uses: actions/cache@v2
        with:
          path: node_modules
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}

      - name: cache build
        uses: actions/cache@v2
        with:
          path: ${{ github.workspace }}/.next/cache
          key: ${{ runner.os }}-nextjs-${{ hashFiles('**/yarn.lock') }}-${{ hashFiles('**.[jt]s', '**.[jt]sx') }}
          restore-keys: ${{ runner.os }}-nextjs-${{ hashFiles('**/yarn.lock') }}-

      - name: yarn build
        run: yarn build:dev

      - name: upload next build
        uses: actions/upload-artifact@v2
        with:
          name: next-build
          path: |
            .next
            public
            dist
          retention-days: 7

  cypress:
    needs: build
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        containers: ['machine I', 'machine II', 'machine III']
    steps:
      - uses: actions/checkout@v2

      - name: cache node modules
        uses: actions/cache@v2
        with:
          path: node_modules
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}

      - name: download next build
        uses: actions/download-artifact@v2
        with:
          name: next-build

      - name: install cypress
        run: yarn cypress install

      - name: cypress
        uses: cypress-io/[email protected]
        with:
          start: yarn start:dev
          wait-on: 'http://localhost:9000'
          parallel: true
          group: 'End-to-End Test Suite'
          record: true
        env:
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title }}
          GOOGLE_MAPS_API_KEY: ${{ secrets.GOOGLE_MAPS_API_KEY }}

[huy-nguyen]

Can this be re-opened? I'm running into the same problem. I'm running only one Cypress test, which passes in GitHub action. The duration of the test is 10 second. The video is 2 minute long but "freezes" (doesn't show any change) after about 10 seconds and I can't see how the test passed.

[huy-nguyen]

Related bug reported in main cypress repo: https://github.com/cypress-io/cypress/issues/19488.

[TildeWill]

Seeing the same issue with Cypress 10 and cypress-io/github-action@v4

[karlhorky]

This is a known issue with Cypress - Cypress cannot take the videos on the default GitHub Actions machines:

• https://github.com/cypress-io/github-action/issues/483
• https://github.com/cypress-io/cypress/issues/16884
• https://github.com/cypress-io/cypress/issues/4722#issuecomment-515562987
• https://github.com/github/roadmap/issues/161

Using the new GitHub Actions "larger runners" is an option, but this may be an undesirable extra cost:

https://github.com/cypress-io/github-action/issues/483#issuecomment-1310424019

[MikeMcC399]

Closing, since this is not an issue with cypress-io/github-action itself.