Add support for jasmine 2

[alextreppass]

jasmine-jsreporter doesn't seem to support jasmine 2.0.0 yet, but @sclevine submitted a PR which seems to have done most of the heavy lifting - see https://github.com/detro/jasmine-jsreporter/pull/7

I can get sauce running the tests by including the modified jasmine-jsreporter inside our Gruntfile's grunt-contrib-jasmine vendor section, which overwrites the JSReporter from grunt-saucelabs, and namespaces window.jasmine.getJSReport().

The screencast then says "625 specs, 0 failures, 9 pending specs" and is green/passing, and the selenium log says all suites and specs are passing, however sauce (or grunt-saucelabs?) is saying the tests haven't passed. Going to the JSUnitTest tab on the sauce job however tells me Jasmine run on Sauce failed: 0 total: 0 passed: 0, finished in s, and Passing 0 tests.

Is Jasmine 2 support on the grunt-saucelabs roadmap? If so what are the blockers currently?

[alextreppass]

I've tried to get the ball rolling with jasmine-jsreporter and jasmine itself - it seems a bunch of information that used to be exposed to reporters in jasmine 1.3.1 is now no-longer available.

https://github.com/detro/jasmine-jsreporter/pull/7

https://github.com/pivotal/jasmine/issues/547

[Jonahss]

the Sauce api gets the JSON data from the old jsreporter and does a lot of parsing on that specific structure to get the data to populate the various fields Sauce uses. The original idea was that everyone already using Jasmine could easily expose their results and Sauce parses it to get what they need. But there's no consistent method for getting the results in Jasmine 2.0 yet.

My preference would be to use the custom framework, explained in the docs. Writing a small script to parse the results into that format would get you most of the data you need, correct? Other people can use the same, or a modified version of the script.

Are there more fields you'd like supported on the custom framework? Or would you prefer to have @sclevine's specific reporter added to sauce's framework support?

[alextreppass]

Ideally if @sclevine's reporter works, have that added to sauce's framework support for when jasmine version is set to 2.0.0.

I'll give custom a play in the coming weeks, see how far I get.

[rkh]

What's the status on this?

[alextreppass]

@rkh - I've made JSReporter with Jasmine 2.0, and have written a JSReporter2 class as part of that project.

Still waiting on Jasmine#575 - the Pivotal guys seem to be MIA for over a month now

[rkh]

For what it's worth, I solved this by using karma.

[alextreppass]

Update: pivotal have merged in the required reporting changes in jasmine 2.0.1 (https://github.com/pivotal/jasmine/pull/575), and jasmine-jsreporter has been updated to match (https://github.com/detro/jasmine-jsreporter/pull/17).

There should no-longer be anything blocking grunt-saucelabs from adopting Jasmine 2.0.1.

Jasmine 2.0.1 and jasmine-jsreporter need to be versioned up in grunt-saucelabs examples folder: https://github.com/axemclion/grunt-saucelabs/tree/b25ff61b7ce0126bbe507f931758ce8cfb64bae7/examples/jasmine/lib

Looks like @stdavis has opened https://github.com/gruntjs/grunt-contrib-jasmine/pull/154 to progress this in grunt-contrib-jasmine

[Jonahss]

just got back from vacation, I'll see about adding this in.

[steveoh]

:+1: we would love to start getting saucy and I believe this is what we are waiting on.

[Jonahss]

thanks for pinging me @steveoh I'll up the priority.

[steveoh]

any updates @Jonahss?

[Jonahss]

Currently working on Appium, maybe if I get my tasks done quick enough I can sneak this in.

[steveoh]

@Jonahss if you tell me what needs to happen I can try to do it.

[Jonahss]

Right now, if you run a unit test and pass in the jasmine framework, Sauce's servers expect to see results in the same format as the test results reporter by the old JasmineReporter.

If you run a nut test and pass in the custom framework, then Sauce's servers are looking for a json object which conforms to the custom framework as outlined in the README.

To add support for a new format, it requires that a Sauce employee add the capability to properly parse the results to the Sauce servers. Unfortunately you can't help with that :( (unless you want a new job)

So your other option is to parse the jasmine2.1 results yourself and get them into either the custom or jasmineReporter formats already supported.

Thanks for continuing to keep this issue active.

[alextreppass]

So if you use jasmine-jsreporter and add JSReporter2 as follows:

jasmine.getEnv().addReporter(new jasmine.JSReporter2())

Results should automatically be populated in the old JasmineReporter format.

(Provided the jasmine 2 reporters haven't changed between 2.0.1 and 2.1)

[lukeapage]

So whats happening with this? Is this supported as long as I use the right reporter?

[Jonahss]

@lukeapage if you follow the instructions given by @alextreppass above, you should be able to use jasmine2.0

[lukeapage]

Thanks, I have done that, was just confused originally as to why this was open if its not an issue.

So far, no luck https://saucelabs.com/jobs/26aa971edb9d4becb74b2c82edc51e86

https://travis-ci.org/less/less.js/jobs/38538939

https://github.com/less/less.js/pull/2236/files

It looks to me like status is blank? But everything else is there? Any suggestions?

[Jonahss]

@lukeapage you might be running into the issue where Sauce can't handle super long test results. Try with a smaller test suite.

[lukeapage]

I read it was 64kb and didn't think we had reached that, but I tried a smaller suite and it worked - thanks. Any idea if that limit will be lifted? Has anyone raised it with sauce? p.s. great repo, it made things super easy

[Jonahss]

Thanks! Most the credit due to all the great collaborators.

Sauce is aware of the limit issue, and it's slated to be fixed. Don't know the deadline though.

On Mon, Oct 20, 2014 at 11:56 PM, Luke Page [email protected] wrote:

I read it was 64kb and didn't think we had reached that, but I tried a smaller suite and it worked - thanks. Any idea if that limit will be lifted? Has anyone raised it with sauce? p.s. great repo, it made things super easy

— Reply to this email directly or view it on GitHub https://github.com/axemclion/grunt-saucelabs/issues/109#issuecomment-59886744 .

[stdavis]

Thanks for the helpful tips @Jonahss. I too ran into the larger test suite. I only have 185 specs which doesn't seem that much to me but running a subset of 14 made everything work. Do you have a link to the bug at sauce that we can track. This is a show stopper for me until I can run my entire test suite.

[Jonahss]

@stdavis it shouldn't be a show stopper. You can just alter your reporting script to only add the info for a failure, and not even bother reporting passes.

[stdavis]

@Jonahss What do you mean by "reporting script"? You mean alter jasmine-jsreporter?

[Jonahss]

Ah yes, altering the jasmine reporter OR just override the call to the reporter and sanitize the results. Sauce Labs runs the following Javascript code to get the results from your page: window.jasmine.getJSReport()

So if you override that method with your own, and call the original method to get and parse the results, you're done. PSEUDO CODE:

var oldFunc = window.jasmine.getJSReport;
window.jasmine.getJSReport = function() {
 var results = oldFunc();
 return removePassingTests(results)
}

Also check out using the custom framework, to have even finer control over what gets reported on Sauce.

[stdavis]

@Jonahss Thanks for the tip. I will implement the work around that you suggest. Really appreciate your help.

[alexan]

I've tried to implement your pseudo code and got it working. This is dependent on jquery

jasmine.getEnv().addReporter(new jasmine.JSReporter2());

(function () {
   var oldFunc = window.jasmine.getJSReport;
   window.jasmine.getJSReport = function () {
      var results = oldFunc();
      if (results) {
         return {
            durationSec: results.durationSec,
            suites: removePassingTests(results.suites),
            passed: results.passed
         };
      } else {
         return null;
      }
   };

   function removePassingTests(suites) {
      return $.map($.grep(suites, grepFailed), mapSuite);
   }

   function mapSuite(suite) {
      return $.extend({}, suite, {
         specs: $.grep(suite.specs, grepFailed),
         suites: removePassingTests(suite.suites)
      });
   }

   function grepFailed(item) {
      return !item.passed;
   }
})();

[leeyeh]

@alexan Thanks, You save my time.

[gordonwoodhull]

FWIW, ES5-only no-jquery version of @alexan's solution for trimming the passing tests:

jasmine.getEnv().addReporter(new jasmine.JSReporter2());

(function () {
    var oldJSReport = window.jasmine.getJSReport;
    window.jasmine.getJSReport = function () {
        var results = oldJSReport();
        if (results) {
            return {
                durationSec: results.durationSec,
                suites: removePassingTests(results.suites),
                passed: results.passed
            };
        } else {
            return null;
        }
    };

    function removePassingTests (suites) {
        return suites.filter(specFailed)
            .map(mapSuite);
    }

    function mapSuite (suite) {
        var result = {};
        for (var s in suite) {
            result[s] = suite[s];
        }
        result.specs = suite.specs.filter(specFailed);
        result.suites = removePassingTests(suite.suites);
        return result;
    }

    function specFailed (item) {
        return !item.passed;
    }
})();