mocha-jenkins-reporter icon indicating copy to clipboard operation
mocha-jenkins-reporter copied to clipboard

Screenshots bug. Filename is not correctly formatted.

Open alexisreina opened this issue 7 years ago • 1 comments

Hi, I'm using this plugin along with protractor in a CI server powered by Jenkins. In the company I work for the requested me to implement a report with screenshots. I can't find much in the docs about it, so I tried myself.

I think I found some issues I'd like to verify with you:

  1. The path is wrong as the option junit_report_path points to a file not to a folder?.
  2. Screenshots are only recorded on failure, but it'll be nice if
  3. It'll be nice if the screenshots filename follow some formatting convention, like not contain spaces use slashes, remove dots and be lowercase, IMO.

In any case, thanks for the reporter, just let you know I'm happy to help If you want. I leave you here

This is how the report looks after I make some tests fail on purpose:

<testsuites name="Voxelcare Integration Tests">
  <testsuite name="Home page" tests="3" errors="0" failures="2" skipped="0" timestamp="2017-08-18T10:57:30" time="29.907">
    <testcase classname="Voxelcare Integration Tests.Home page" name="should be at presentation" time="2.692">
      <failure message="expected &apos;http://192.168.99.100:8282/#!/content/presentation&apos; to equal &apos;http://192.168.1.21:8282/#!/content/presentation&apos;">+http://192.168.1.21:8282/#!/content/presentation
-http://192.168.99.100:8282/#!/content/presentation
</failure>
      <system-out>[[ATTACHMENT|D:\fuentes\web\webBase1\voxelcareang\tests\results\vx-release-test-results.xml\Home pageVoxelcare Integration Tests.Home pageshould be at presentation.png]]</system-out>
    </testcase>
    <testcase classname="Voxelcare Integration Tests.Home page" name="should contain an iframe with the landing page" time="1.003">
    </testcase>
    <testcase classname="Voxelcare Integration Tests.Home page" name="should be able to navigate to the login page" time="15.009">
      <failure message="Timeout of 15000ms exceeded. For async tests and hooks, ensure &quot;done()&quot; is called; if returning a Promise, ensure it resolves."></failure>
      <system-out>[[ATTACHMENT|D:\fuentes\web\webBase1\voxelcareang\tests\results\vx-release-test-results.xml\Home pageVoxelcare Integration Tests.Home pageshould be able to navigate to the login page.png]]</system-out>
    </testcase>
  </testsuite>
  <testsuite name="Login page" tests="1" errors="0" failures="1" skipped="0" timestamp="2017-08-18T10:58:00" time="1.435">
    <testcase classname="Voxelcare Integration Tests.Login page" name="should be at login" time="0.293">
      <failure message="expected &apos;http://192.168.99.100:8282/#!/content/order/login&apos; to equal &apos;http://192.168.1.21:8282/#!/content/order/login&apos;">+http://192.168.1.21:8282/#!/content/order/login
-http://192.168.99.100:8282/#!/content/order/login
</failure>
      <system-out>[[ATTACHMENT|D:\fuentes\web\webBase1\voxelcareang\tests\results\vx-release-test-results.xml\Login pageVoxelcare Integration Tests.Login pageshould be at login.png]]</system-out>
    </testcase>
...
...

This is the protractor config file:

var baseUrl = 'http://192.168.1.21:8282/';

exports.config = {

    framework: 'mocha',

    directConnect: true,

    baseUrl: baseUrl,

    specs: ['../**/*_specs.js'],

    capabilities: {
        'browserName': 'chrome'
    },

    mochaOpts: {
        reporter: 'mocha-jenkins-reporter',
        reporterOptions: {
            junit_report_name: 'Voxelcare Integration Tests',
            junit_report_path: './tests/results/vx-release-test-results.xml',
            screenshots: 'spec',
        },
        slow: 3000,
        timeout: 15000
    },

    params: {
        baseUrl: baseUrl,
    },

    // Use the external Chai As Promised to deal with resolving promises in
    // expectations
    onPrepare: function() {
        expect = require("chai").use(require("chai-as-promised")).expect;
    },

    ignoreUncaughtExceptions: true

};

alexisreina avatar Aug 18 '17 11:08 alexisreina

As I expected the generated path for the 'spec' screenshots is wrong, giving the next error in Jenkins cause the system is trying create a folder with named as the xml report, including the extension.

Error: EEXIST: file already exists, mkdir '/var/jenkins_home/workspace/developIntegrationTests/voxelcareang/tests/results/vx-release-test-results.xml'
at Error (native)

While the documentation of the Jenkins plugin states that should be in the same folder, I paste you the example found on Wiki of the plugin

  • test report in .../target/surefire-reports/TEST-foo.bar.MyTest.xml
  • test class is foo.bar.MyTest
  • test attachment directory: .../target/surefire-reports/foo.bar.MyTest/

There's another issue with the name of the file. As you can see in the example report I pasted in my other comment the name is not well formatted, there's a missing space between the suite name and the test name in the image name.

Login pageVoxelcare Integration Tests.Login pageshould be at login.png

Plus when you not declare a name it repeats the suit name. There's a missing blank space here, but, why not just use junit_report_name + ' ' + currenTest.fullTitle() + imagetype so I get something like: Test Results. Login Page should be at login.png intead of Test results.Login Pageshould be at login.png

https://github.com/juhovh/mocha-jenkins-reporter/blob/master/lib/jenkins.js#L143

alexisreina avatar Aug 18 '17 13:08 alexisreina