codeceptjs-resemblehelper icon indicating copy to clipboard operation
codeceptjs-resemblehelper copied to clipboard

Setting needsSameDimension flag to false isn't validating the difference of text between two locales.

Open sharath2106 opened this issue 2 years ago • 5 comments

We are using codeceptjs-resemblehelper to perform visual regression testing for our application that supports multiple locales. We are logging in with users mapped to these locales to achieve this.

Since our page is dynamic and it contains data. We are using I.seeVisualDiffForElement method and find the diff only for the elements that are static in the application. Some translations are rendering elements tad bigger than others.

On executing a negative case where the user belongs to the locale en-GB and compares the image for fr-FR, we see the below error - The base image is of 65 X 156 and actual image is of 62 X 156. Please use images of same dimensions so as to avoid any unexpected results.

searchBar-en-GB

searchBar-fr-FR

In order to ignore image size, we used the attribute needSameDimensions: false. But unfortunately, after using this attribute in our tests, the screen comparison is not working. The diff is not considered, and the tests are passing even though the locale is completely different.

Can you take a look at this issue?

sharath2106 avatar Aug 03 '22 09:08 sharath2106

Hi @sharath2106 I will take a look at this issue, seems like a bug to me. thank you for reporting.

puneet0191 avatar Aug 03 '22 10:08 puneet0191

Sure, I can't recall exactly which part causes this but feel free to send a patch for fixing this.

puneet0191 avatar Aug 04 '22 08:08 puneet0191

@sharath2106 sorry I could not find enough time to verify, send PR, we will have it tested and make a new release.

puneet0191 avatar Aug 15 '22 15:08 puneet0191

Hi @puneet0191 . I feel the issue is not because of the if statement. My fix which i was talking about was on this statement, but i think it's not.

if (options.needsSameDimension && !data.isSameDimensions) {
            let dimensions1 = sizeOf(baseImage);
            let dimensions2 = sizeOf(actualImage);
            reject(new Error(`The base image is of ${dimensions1.height} X ${dimensions1.width} and actual image is of ${dimensions2.height} X ${dimensions2.width}. Please use images of same dimensions so as to avoid any unexpected results.`));
          }

Not sure how data.isSameDimensions is being fetched. Could you take a deeper dive on this?

sharath2106 avatar Sep 27 '22 10:09 sharath2106

@puneet0191 Also, ignoredBox is not taking the precedence over sameDimensions -false flag. I think, this need to be looked and sorted out for our scripts to work.

sharath2106 avatar Sep 29 '22 08:09 sharath2106