Multiple browser tests running simultaneously with same screenshot name fails

[kmturley]

I think there is an issue with tests running on multiple browsers simultaneously if they use the same name. Here is my setup:

wdio.conf.js

capabilities: [{
    browserName: 'chrome'
}, {
    browserName: 'firefox'
}, {
    browserName: 'internet explorer'
}]

header.spec.js

describe('header', function () {
    it('should look the same', function (done) {
        browser
            .url('/')
            .webdrivercss('home', {
                name: 'header',
                elem: '.header'
            }, function (err, res) {
                expect(res.header[0].isWithinMisMatchTolerance).toBe(true);
            }).call(done);
    });
});

One browser always fails because they are sharing the same screenshot filename and running at the same time:

• home.320px.png
• home.header.320px.png

However if I make the name to be unique per browser. It seems to work every time:

describe('header', function () {
    it('should look the same', function (done) {
        browser
            .url('/')
            .webdrivercss('home.' + browser.desiredCapabilities.browserName, {
                name: 'header',
                elem: '.header'
            }, function (err, res) {
                expect(res.header[0].isWithinMisMatchTolerance).toBe(true);
            }).call(done);
    });
});

This then creates separate files per browser, which makes more sense to me:

• home.chrome.320px.png
• home.chrome.header.320px.baseline.png

Shouldn't this naming be done automatically by webdrivercss? Otherwise it fails for anyone using multiple browsers!

If you add it to webdrivercss, I would suggest we move the browser name to the end of the filename, after the user inputted name e.g.:

• home.header.chrome.320px.baseline.png
• home.header.firefox.320px.baseline.png

This allows screenshots of similar types to be grouped together, and you can easily see browser screenshots side by side.

Another point to notice is that "internet explorer" has a space in the browser name, this causes issue when naming screenshots. I would suggest using either:

• internet-explorer
• ie

[JamEngulfer]

I use a folder system that puts screenshots into different folders based on metadata. That works pretty well.

[kmturley]

@JamEngulfer Nice, can you post an example of how that works? does it fix the bug mentioned above? or a workaround?

[JamEngulfer]

@kmturley Well, it's a pretty simple solution. I just generate a file path that takes into account things like the browser, element and url, then set it with the screenshotRoot option.

[kmturley]

OK? that didn't really answer my questions though!? :)

[JamEngulfer]

Sorry, yeah. You're right.

From what I can see, you haven't really got a bug there, it's just reusing the same folder (unless you are actually making it use different folders).

You can set the folder that the screenshots are saved in with screenshotRoot: <folder> in the object you pass to webdrivercss.init(client, options).

When I run each of my tests, I have an object with all of the information like browser, url and element. I just use the built in path module's join method to quickly make a folder path from this information. I then use this in the aforementioned screenshotRoot option.

If I haven't been clear or if you need more information, just ask.

[kmturley]

That sounds like a good approach, but does that mean you need to init() for each browser separately? to be able to change the screenshotRoot per browser? or you pass a dynamic variable?