cypress
cypress copied to clipboard
cypress-io
Out of memory issue after update a browsers to version 104+ (Chrome&Edge)
Current behavior
We have configured build to run UI tests on both Edge and Chrome browsers. Last time I noticed that on Edge test are not executed and finished after 6h (pipeline timeout). Now we have the same situation on Chrome, that test are not executed. Without any error/log etc.
What interesting both starting to timeout after update Edge from version 103 to 104 and Chrome from version 103 to 104. No any code changes between update from 103 to 104, so same code base failing after update to 104.
I run it locally and always have "Out of memory" issue screen during execution. --headed flag added to investigate I try it on different versions, and always have same issue.
Desired behavior
Same code should working on browsers version 103 and 104.
Test code to reproduce
No any code, test are not running, because of out of memory issue.
Cypress Version
9.3.1, 9.7.0, 10.0
Node version
16.16.0
Operating System
Windows Azure Virtual Machine
Debug Logs
No response
Other
No response
Also seeing the same issue, no major code changes (nothing that would cause any sort of memory leak) and only the updated versioning of browsers. Debugging was always an issue for us with memory crashes but now we are seeing every agent failing to run a test in our azure pipelines due to what we suspect now is a memory issue with the chrome browser being used. Switched to Electron in the same pipeline and we had no issues
I haven't issues with electron so far. Till update Electron to version 94. After that same issue occurs for all browsers. Super weird.
I'm having the same issue with chrome 104 and chrome 105 beta, edge and other latest chromium versions.
I'm having the same issue. What I try to do at the moment is to atomize as much as possible each spec, by making spec change it seems that the memory is "reset" and the problem does not occur, but even if there is only one test case and it is too long, the problem will occur. The most tedious thing is that it doesn't jump any error, it just freezes.
Following up with a conversation on the Cypress Discord server, I'm running Chrome 104 with Cypress 9.5.4, running headed, and I'm not having this issue, though others definitely are. The folks who are mentioned that one of them was running 9.6.1, which was receiving the OOM behavior.
My longest spec is 50+ test cases, some of which are individually 50+ lines long for reference, in case spec size is a factor.
If there's any further info I can provide please let me know.
Same issue, chrome 104, same with electron, same like previous versions of cypress or browsers:
- long scenarios takes up the browser's memory and crashes.
- Cypress 9.6 was the same
- Now running Cypress 10.6.0 - same
Tried every option available in github issues such as:
- setting chrome memory allocation unsuccessfully
- numTestsKeepInMemory: 1 - no use
It would be nice if we could config chrome at least to take up more memory from the computer. :(
PS: Headless is not affected, only headed runs.
Hi @MusolQA ! Thank you for reporting this issue, I'm sorry to hear you and others are running into this problem. In order to investigate, we'll need to be able to reproduce on our end. Can you set up a cypress-tiny-test that reproduces the issue? Thanks
@rachelruderman
Thanks for your answer. https://github.com/MusolQA/cypress-test-tiny
Here repository with code that causes Out of memory issue. Please install dependencies, I reproduced in on my local Windows10 machine.
To run please install dependencies (preprocessor to run BDD), and run test.feature file. Below my results.
I add a bunch of When, Then steps in feature file, but basically its failing after ~2-3 mins of each scenario execution.
Headless mode from CMD
Open via Cypress GUI
Btw. I can see that similar issue was reported and solved in January, here: https://github.com/cypress-io/cypress/issues/17853 Seems that its back, maybe it help you where looking for a problem. I think it should be major priority, cause we are stuck due to issue.
Btw. I can see that similar issue was reported and solved in January, here: #17853 Seems that its back, maybe it help you where looking for a problem. I think it should be major priority, cause we are stuck due to issue.
@MusolQA That was reported by me, and it's not the same issue. Issue#17853 was caused due to changes in cypress 8.2 and above while this new issue is caused due to changes in memory usage for Chrome 104 and 105 beta. I do however agree that it should be considered a high priority since we are having to run tests on chrome 103 and aren't able to run tests on chrome 104 and 105 beta versions
it is due to the new browser versions. I didn't change anything in my code but after the browser update to Chrome 104, many test specs crashed. I downgraded to Chrome 102 and the problems stopped.
Yes, definitely problem is between cypress and latest chromium relation. However you know, downgrade the browsers version might be workaround for few days, but tests should be run on latest version to be reliable.
See one of the several issues that have been reported on google support for chrome 104 memory. https://support.google.com/chrome/thread/174889666/increasing-the-memory-limit-on-chrome-v104?hl=en
@rachelruderman any update or progress?
i don't think it's Cypress, but a problem with the Chrome browser, as you can see one post above.
And it's final answer? Due to this chrome change and limitation we cannot run tests with take more 2mins on Chrome/Edge? As a workaround we can use Firefox that doesn't have this issue with memory
At least Cypress should fail the test case when the memory problem occurs. At this point it just hangs.
Potentially relevant Chromium issue: https://crbug.com/1353749
Edit: and another one: https://crbug.com/1352649
Hi @MusolQA , thank you for creating the cypress-tiny-test repro! I ran it on my personal machine (Mac OS) with Chrome 104 and it passed. Can you tell me a bit more about your Windows Azure Virtual Machine? What OS version are you on and how much memory does it have? TIA!
@rachelruderman , Thanks. Super weird that on MacOS its working fine...
Below our Azure VM configuration:
- OS Microsoft Windows Server 2019 10.0.17763 Image: windows-2019 Version: 20220821.1 https://github.com/actions/runner-images/blob/win19/20220821.1/images/win/Windows2019-Readme.md
- RAM t2.2xlarge - which equates to 32G RAM
Same issue.
Sometimes it crashes with an out of memory error, sometimes it just freezes (even on the last step of a running test. the moment it should close, having almost succeeded to finish).
Tried to separate into multiple specs and handle more timeouts/waits, disabling the dom snapshot autosaving to 0, these helped to reduce the problem but it still exists.
Memory related problems initially started months ago after a version update.
We recently upgraded from v9.7 to v10.7 and ran into this issue. This happens in both headless and headed mode. I reproduce this locally on my macOS system but I also see the failures on our CI (Docker w/ node base image).
The repro rate for me is ~50%. It usually happens on the second repeat of a full spec, but if I do not close the crashed window, and instead click "Reload" button it will always crash on subsequent runs. This further corroborates the theory that this is a memory issue unless the process is killed.
In a spec with 6 tests I'm able to reproduce when running 6 specs, less often when skipping one of them and I wasn't able to reproduce when running 4 or fewer specs. The crash always happens on the 6th step.
Chrome v105.0.5195.52 - reproducing reliably Chrome Canary v107.0.5274.0 - reproducing reliably Electron v102.0.5005.148 - not reproducing Firefox v99.0.1 - not reproducing
Are there any Chrome/Cypress debug logs I can enable to be of help here?
The chrome memory issue will likely be fixed in 106 https://bugs.chromium.org/p/chromium/issues/detail?id=1352649
FYI: https://github.com/cypress-io/cypress/issues/21135#issuecomment-1235643009
@emilyrohrbough
I'm just wondering and I just created a same automation test in Playwright for UI Automation, and also use Chrome as a browser to execute and same hardware to run. In this case I don't have any issues with memory, what can suspect that its more Cypress related issue?
@PawelRuszlewicz Comparing Cypress to Playwright is not 1:1. Playwright does not inject code into the browser to test your page, where as Cypress is injected in the browser when testing your pages -- the reason Cypress can provide Time Travel and dom snapshots for commands that have executed and it's corresponding logs. I have no doubt Cypress is impacting the amount memory used in Chrome, but also wouldn't to say this is the only issue here -- though it could be.
Hope this fix applies to older cypress versions as we are using cypress 9.5.4 and Chrome 105, we are getting this issue. We are not planning to update our suite to 10+ yet.
