Videos don't seem to work on GitHub Action default runners

[webbertakken]

Context

Bug report regarding videos on GitHub Actions default runners.

Steps

I have just set up cypress for my repository.

Everything seems to be set up correctly, and yarn cypress run yields a video that works.

Now I'd like to reproduce this in CI, and created a workflow with the same settings apart from the URL:

Relevant part of the workflow

      - name: Run Cypress 🌲
        uses: cypress-io/github-action@v2
        env:
          CYPRESS_BASE_URL: ${{ github.event.deployment_status.target_url }}
      - name: Upload video 📼
        uses: actions/upload-artifact@v2
        if: always()
        with:
          name: Test videos
          path: cypress/videos
          retention-days: 14

I'm using default ubuntu-latest runner with default settings.

** Results**

The tests run fine (all 91 succeed as expected) and the video uploads. However the video cuts out after a few seconds. Only the last frame is updated to show the tests. Presumably because the machine runs out of resources while loading the page.

Video artifact

Test videos.zip

Video file

https://user-images.githubusercontent.com/20756439/150003505-a2fcf6b4-08da-4596-925b-9e9cb2dd473b.mp4

Screenshot of last frame

Related

I believe that most reports here are also related to this issue.

[henrikq]

I'm experiencing the same thing.

[MartijnHols]

FWIW the GitHub runners' CPUs are terribly slow, and disabling videos entirely was a big performance gain for me (about 50%). While videos are very valuable, the massive run time reduction was worth it for me. It also made my tests less flaky, as there are fewer timeouts.

This CPU bottleneck might also be why Cypress has a hard time recording and encoding the videos.

https://github.com/github/roadmap/issues/161 is scheduled for Q1 of this year, hopefully this will make Cypress videos viable.

[henrikq]

@MartijnHols How did you disable video?

[webbertakken]

For me it doesn't matter too much if it goes slower, as long as it works. Is there no way to throttle the tests more in order to get the video to work somehow?

Personally I don't feel disabling video entirely is a good solution to this issue.

[MartijnHols]

@MartijnHols How did you disable video?

https://docs.cypress.io/guides/guides/screenshots-and-videos#Videos

[alessioerosferri]

FWIW the GitHub runners' CPUs are terribly slow, and disabling videos entirely was a big performance gain for me (about 50%). While videos are very valuable, the massive run time reduction was worth it for me. It also made my tests less flaky, as there are fewer timeouts.

This CPU bottleneck might also be why Cypress has a hard time recording and encoding the videos.

github/roadmap#161 is scheduled for Q1 of this year, hopefully this will make Cypress videos viable.

Are you running the tests in parallel? I tried disabling videos but I am getting this issue #507

[jennifer-shehane]

This is a CPU issue. There is not enough CPU to run Cypress, record video and whatever else is running in the GHA. Default CPU for GHA: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources

CPU Recommendations:

• 2 CPUs minimum, for Cypress (Node + Browser)
• 1 additional CPU if video recording is enabled
• 1 additional CPU per process you run outside of Cypress, such as:
  • App server (frontend)
  • App server (backend)
  • App database
  • Infra component (redis, etc..)

[karlhorky]

Wonder if switching to the macOS runners would help (eg. macos-latest or macos-12):

Hardware specification for macOS virtual machines:

• 3-core CPU
• 14 GB of RAM
• 14 GB of SSD space

[karlhorky]

Looks like GitHub has a beta of larger GitHub Actions runners 🙌 : https://github.blog/changelog/2022-09-01-github-actions-larger-runners-are-now-in-public-beta/

[karlhorky]

@jennifer-shehane I guess you would recommend at least the 4-cores option here? Or would the minimum be 8-cores instead?

[admah]

@karlhorky based on our hardware recommendations 4-cores should be fine.

[karlhorky]

Great, I have requested access to the new larger GitHub Actions runners for installation to our organization and will try the videos on a 4-core runner. I'll report back here with what I find.

[karlhorky]

Ok, finally got a chance to try GitHub Actions larger runners out and it seems that it still freezes / stutters multiple times on a 4-core runner with 16GB RAM (Ubuntu). This is with [email protected].

The 8-core / 32GB RAM runner produced almost smooth videos (stuttered a bit at the beginning).

Wonder if anything else can be done on Cypress' side to make it more efficient.

cc @admah @jennifer-shehane

[MikeMcC399]

@webbertakken

This issue has been discussed and the opinion was that the problem with video capture was due to hardware capacity.

In the meantime, as of Cypress 13.0.0, video capture in Cypress is disabled by default and at the same time Test Replay on Cypress Cloud has been introduced.

github-action cannot correct the hardware issue of video capture, so I would suggest to close this issue now.

[karlhorky]

For anyone seeing this issue at some future point in time, just switch to Playwright - videos work out of the box on default GitHub Actions runners with no problems with hardware capacity or anything (and Playwright also solves many other Cypress-specific problems)

[MikeMcC399]

@karlhorky

• That's interesting. Does Playwright also capture videos for Firefox - something that Cypress can no longer do (https://github.com/cypress-io/cypress/issues/18415)

[karlhorky]

• Does Playwright also capture videos for Firefox

haven't tried yet, but Playwright supports Firefox, so I would guess so

[MikeMcC399]

Closing, since this is not an issue which can be resolved by github-action.