visual-testing icon indicating copy to clipboard operation
visual-testing copied to clipboard

screenshot not saving; comparison always fails

Open lvxwrkr777 opened this issue 3 years ago • 7 comments

NOTE: Be aware of the fact that this module is only a plugin that provides all functionality from webdriver-image-comparison. So please check if this is an issue with this plugin or with webdriver-image-comparison.

Environment (please complete the following information):

  • Node.js version: v14.18.2
  • NPM version: 6.14.15
  • Browser name and version: Chrome 97
  • Platform name and version: Windows 10
  • WebdriverIO version: 7.16.13
  • wdio-image-comparison-service version: 3.1.0

Config of WebdriverIO + wdio-image-comparison-service

services: [
        ['chromedriver',{}],
        ['image-comparison',
        {
            baselineFolder: join(process.cwd(), './tests/Baseline/'),
            formatImageName: '{tag}-{logName}-{width}x{height}',
            screenshotPath: join(process.cwd(), '.tmp/'),
            savePerInstance: true,
            autoSaveBaseline: true,
            blockOutStatusBar: true,
            blockOutToolBar: true,
        // NOTE: When you are testing a hybrid app please use this setting
            isHybridApp: true,
        // Options for the tabbing image
            tabbableOptions:{
                circle:{
                    size: 18,
                    fontSize: 18,
                // ...
            },
                line:{
                    color: '#ff221a', // hex-code or for example words like `red|black|green`
                    width: 3,
            },
        }
        // ... more options
    }],
    ],

Describe the bug It seems that no baseline images are ever saved, so the comparison always fails

To Reproduce

    xit('Screenshot of vendors page taken', async () => {
        browser.saveFullPageScreen('fullPage');
    });
    it('Screenshot matches baseline', async () => {
        expect(browser.checkFullPageScreen('fullPage')).toEqual(0);
    });

Expected behavior I should be able to take/save screenshots, but it doesn't appear to be saving anything to baseline or any tmp folder, so of course the comparison fails.

Log

[chrome 97.0.4692.99 windows #0-0] expect(received).toEqual(expected) // deep equality

Expected: 0
Received: {}

Additional context My tests are in TS. Yes, I have added the comparison service types to my TS config.

lvxwrkr777 avatar Feb 04 '22 21:02 lvxwrkr777

wdio-image-comparison-service by default saves the baseline image in ./.tmp/actual. If you remove baselineFolder: join(process.cwd(), './tests/Baseline/') from wdio.conf.js file and update it block of image comparison as mentioned below

it('Screenshot matches baseline', async () => {
const testOptions = {
    actualFolder: path.join(process.cwd(), './.tmp/checkActual'),
    baselineFolder: join(process.cwd(), './.tmp/actual'),
    diffFolder: path.join(process.cwd(), './.tmp/testDiff'),
    returnAllCompareData: true,
 };

expect(browser.checkFullPageScreen('fullPage', testOptions)).toEqual(0);
});

Then it works

mayankshukla94 avatar Feb 15 '22 12:02 mayankshukla94

I still got:

`[chrome 97.0.4692.99 windows #0-0] expect(received).toEqual(expected) // deep equality

Expected: 0 Received: {} [chrome 97.0.4692.99 windows #0-0] Error: expect(received).toEqual(expected) // deep equality [chrome 97.0.4692.99 windows #0-0] [chrome 97.0.4692.99 windows #0-0] Expected: 0 [chrome 97.0.4692.99 windows #0-0] Received: {}`

lvxwrkr777 avatar Feb 16 '22 19:02 lvxwrkr777

Hi @lvxwrkr777

I'm currently on vacation and can't verify it, I'll check it when I'm back. I have the feeling this is related to Windows only because on Mac/Linux this just pases

wswebcreation avatar Feb 20 '22 06:02 wswebcreation

@wswebcreation Any update on this?

mayankshukla94 avatar Apr 01 '22 17:04 mayankshukla94

Encounter similar issue, I guess there no fix on this?

rasis2 avatar Aug 22 '23 02:08 rasis2

Can someone provide a minimal reproducible example as GitHub project I can clone and use to investigate?

christian-bromann avatar Jan 05 '24 04:01 christian-bromann

Can someone provide a minimal reproducible example as GitHub project I can clone and use to investigate?

I'ma call you 'Christian Brogrammer', if you can't figure this out.

I don't work for the company where I had to operate within that context anymore, but it was an MS-DOS environment with node.js. Weird, I know. I found some port bugs in modules originally written for Linux.

lvxwrkr777 avatar Jan 08 '24 19:01 lvxwrkr777