Screenshot height changes between runs.

[steveharrison]

Note: I originally raised this issue in the Cypress repository, but they think that it is an issue with cypress-image-snapshot instead.

Current behavior:

When I run npx cypress run --browser chrome in my example project, example.zip, it generates different sized screenshots on multiple runs, with no files changed in between runs. I am running the command in a Terminal window on my external monitor for all runs, so the display resolution should be consistent.

Last week, it produced a screenshot that was 1000x660px (the "Generated 1" image in the example project). I cleared the baseline image, ran it again, and got a 1000x487px screenshot (the "Generated 2" image in the example project). I ran the tests again, and it failed, because it was now comparing it with a 1000x537px screenshot. I cleared the baseline image again, ran the command again, and got a 1000x537px screenshot. I ran the tests again, and they succeeded. It seems like most of the time, the command produces a 1000x537px screenshot, but sometimes, it produces something different.

Firstly, I'm confused as to why the height changes. Secondly, I'm confused as to where the 537px value is coming from, since I'm setting the browser size to 1000x600 in the before:browser:launch hook in cypress/plugins/index.js.

If you think this is instead an issue with the cypress-image-snapshot plugin, I am happy to raise it there instead.

Desired behavior:

I would like the screenshots to be to be able to run npx cypress run --browser chrome multiple times under the same conditions and have resulting screenshots the same size.

Test code to reproduce

I've attached a ZIP file. You just need to unzip it, use Node 13.9.0, run npm i, run npm start, and then run npx cypress --run browser chrome.

Versions

Node 13.9.0 Cypress 4.9.0 macOS 10.15.3

[jfhector]

I've been struggling with the same issue. In my case, the width and height of the screenshots are both off by just 1px.

@jaredpalmer Thanks for this useful library. I'd love to make use of it at my workplace, but I still need to resolve this issue. I'm mentioning you so it's on your radar.

@steveharrison did you manage to find a workaround to this issue?

[jfhector]

Actually, my issue had to do with: https://github.com/jaredpalmer/cypress-image-snapshot/issues/95 Which is a Cypress issue / unexpected behaviour.

[qcn]

I've been regularly encountering exactly the same issue, with the 50px height difference. My guess is that it could be something to do with page reflow when hiding the test-runner interface (there's a toolbar at the top of the viewport), which would make it a Cypress issue rather than a cypress-image-snapshot issue. I have the same issue whether it's in viewport or fullPage mode.

[mkgarg159347]

I had similar issues where the tests were failing due to different image sizes in each run. After a struggle, I found that there is a property named " allowSizeMismatch?: boolean;" which can be passed to overcome this issue. I set it up in index.js file as

addMatchImageSnapshotCommand({ allowSizeMismatch: true, // allow the image size to be diff capture: 'viewport', // capture viewport in screenshot });

[lexanth]

NB: If using the allowSizeMismatch option, if the size mismatches and the diff goes over the threshold, this plugin will only report the size mismatch.

https://github.com/jaredpalmer/cypress-image-snapshot/blob/master/src/command.js#L46-L55 is the line which causes the issue - should probably be

            const message = diffSize && !options.allowSizeMismatch
              ? `Image size (${imageDimensions.baselineWidth}x${
                  imageDimensions.baselineHeight
                }) different than saved snapshot size (${
                  imageDimensions.receivedWidth
                }x${
                  imageDimensions.receivedHeight
                }).\nSee diff for details: ${diffOutputPath}`
              : `Image was ${diffRatio *
                  100}% different from saved snapshot with ${diffPixelCount} different pixels.\nSee diff for details: ${diffOutputPath}`;

[Pieras2]

I don't know if the problem is with size mismatch. In my case I see some additional stripe which changes image captured vs the real one. In my opiniom the problem is with cypress, not this plugin. This plugin just calls cy.screenshot() and this cypress command does the view switch and it does the image too quick

[shdb2]

I had a similar problem: Some (but not all) image snapshots in the pipeline had a slightly different size than locally but only with a few pixels: Error: Image size (1080x114) different than saved snapshot size (1080x112)., Error: Image size (1037x227) different than saved snapshot size (1037x226) and not all images. The images with the wrong size had a line with a few pixels at the bottom that should not belong there (maybe the same as the additional stripe of @Pieras2?).

Local Environment: Windows with node v14.17.6, Cypress 9.0.0 and Chrome 96 (headless) Pipeline Environment: using Docker image cypress/browsers:node14.17.0-chrome91-ff89

Both times using cypress run --browser chrome

I don't know why but for me the issue in the pipeline was fixed when I called cy.screenshot() directly before cy.matchImageSnapshot():

// workaround to prevent screenshots of the wrong size in pipeline
cy.dataCy('someId').screenshot('my-image');
cy.dataCy('someId').matchImageSnapshot('my-image');

Tested with cypress-image-snapshot 4.0.1.

[Pieras2]

Yes, this is exactly the same issue. Whatever it sounds I am happy that someone confirms my finding.

In my opinion it shall be fixed on Cypress site, not the plugin. Cypress does the screenshot action too fast. At least some parameter is added to cy.screenshot to make the screenshot after some timeout. We. Am adjust it the way we need. 100 or even 300ms is mot that much to wait a bit longer so that the browser manage to finish rendering screen

śr., 17 lis 2021, 17:46 użytkownik shdb2 @.***> napisał:

I had a similar problem: Some (but not all) image snapshots in the pipeline had a slightly different size than locally but only with a few pixels: Error: Image size (1080x114) different than saved snapshot size (1080x112)., Error: Image size (1037x227) different than saved snapshot size (1037x226) and not all images. The images with the wrong size had a line with a few pixels at the bottom that should not belong there (maybe the same as the additional stripe of @Pieras2 https://github.com/Pieras2 ?).

[image: image] https://user-images.githubusercontent.com/93133776/142244121-a3c5bccb-fda8-4efe-b57b-69ca5d3b644a.png

Local Environment: Windows with node v14.17.6, Cypress 9.0.0 and Chrome 96 (headless) Pipeline Environment: using Docker image cypress/browsers:node14.17.0-chrome91-ff89

Both times using cypress run --browser chrome

I don't know why but for me the issue in the pipeline was fixed when I called cy.screenshot() directly before cy.matchImageSnapshot():

// workaround to prevent screenshots of the wrong size in pipeline cy.dataCy('someId').screenshot('my-image'); cy.dataCy('someId').matchImageSnapshot('my-image');

Tested with cypress-image-snapshot 4.0.1.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jaredpalmer/cypress-image-snapshot/issues/137#issuecomment-971762030, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUVMH6ZX2MARJKDEAZX753UMPL7VANCNFSM4OT5ZPQQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[shdb2]

Okay seems like I ran out of luck today. Using cy.screenshot(); did not work anymore today. But I found that full-screen screenshots seem to work better, so instead of screenshotting a single element using

cy.dataCy('someId').matchImageSnapshot('my-image');

I now use

cy.matchImageSnapshot('my-image');

Additionally, the failing tests had fonts in them that were rendered differently in Windows and Linux which added another layer to the problem (subpixel rendering) but I was able to handle that using thresholds.

I don't like that version that much because now the tests are not that independent as I would like them to be but I ran out of ideas and just want my tests to work consistently.

[viktorgogulenko]

I have the same issue even with latest 12.8.1 version of Cypress - locally with Electron browser I'm making screenshots and when it runs on Docker environment in CI, tests are failing with error: Warning: Images size mismatch - new screenshot is 1280px by 695px while old one is 1280px by 692 (width x height). So by some reason it just slightly changing image only in one dimension.