support reporting for CI servers like Hudson, Bamboo, and Cruise

[ghost]

trafficstars

Add support for CI servers like Bamboo or Hudson or Cruise by integrating the junit reporter for jasmine from https://github.com/larrymyers/jasmine-reporters. Junit output format is key.

[asalant]

I would like to see this too.

[christiannelson]

Same here. We use TeamCity and it supports a number of formats out of the box, including test::unit, rspec, cucumber, test-spec, shoulda, and nUnit. If there's a hook for formatting output, it'd be fun to write a formatter and make it available for others to use.

[jeffremer]

Junit output format is key

+1

[morganchristiansson]

+1

[johnbintz]

OK, I finally got around to this and it's all merged into HEAD. You can use RSpec-like formatter options for writing to different files with different reporters:

jasmine-headless-webkit -f File:report.txt -f Tap:tap-output.txt -f Console

From that small example, you can see there are three built-in reporters:

• File: the writer for Jasmine::Headless::ReportFile files
• Tap: Tap output
• Console: The output destined for the console.

Take it for a spin. It's still kind of young and needs more eyes on it.

You can bring in Teamcity and JUnit reporters by including the jasmine-headless-webkit-reporters gem:

# Gemfile

gem 'jasmine-headless-webkit', :git => 'git://github.com/johnbintz/jasmine-headless-webkit.git'
gem 'jasmine-headless-webkit-reporters', :git => 'git://github.com/johnbintz/jasmine-headless-webkit-reporters.git'

That gem, once included, lets you do:

jasmine-headless-webkit -f JUnit:output.xml -f Teamcity:teamcity.txt

[christiannelson]

@johnbintz Excited to see support for the new formatters! Just gave it a try... two observations so far:

jasmine-headless-webkit-reporters uses a git submodule to pull in from jasmine-reporters. That submodules had to be manually init/updated before using any of the reporters. That doesn't make it super easy to use.

If I specify just the teamcity or junit reporter and leave off console (e.g. jasmine-headless-webkit -f JUnit:junit.xml), the runner fails with this error "WARNING: too many runs and the test is still not finished!". Adding -f Console to the command line makes everything happy.

Now let's see what our build server can do with the output.

[christiannelson]

@johnbintz That error also occurs with the params "-f Teamcity", which dumps the output to the console. It works awesome until teamcity thinks there's a failure (because of the return code).

[johnbintz]

@christiannelson Per your comments:

For actual releases, the submodule will be pulled in for release. This was the same procedure the Jasmine project was using for making sure the Jasmine JS libraries were in sync. Since it's highly unlikely the dependent project will ever gemify their reporters like Jasmine did with Jasmine core, this approach will have to do for now. It's either this or copy down the scripts via a Rake task, and if Rake becomes a better solution, I'll switch to that.

For the timeouts, the JHW wrappers need to ping the runner to let it know the test suite is alive and running specs, which means I'll have to extend one of the methods on the parent original runners. I'll do that soon.

[johnbintz]

Try the latest HEAD of jasmine-headless-webkit-reporters. The reporters shouldn't fail on long runs anymore, and should properly shut down when the tests are done.

[christiannelson]

@johnbintz that fixed the "busy" problem and teamcity is now reporting tests as expected.

I found another anomaly with HEAD: guard-jasmine-headless-webkit reports the error "spec runner interrupted" (via growl) on every run. It happens whether or not the specs all pass.

[christiannelson]

I created a new issue for the spec runner interrupted bug (#105).

[andreacfm]

I get an error launching:

jasmine-headless-webkit -f JUnit:junit.xml

Error is:

[jasmine-headless-webkit] Not an asset: nil (Jasmine::Headless::InvalidUniqueAsset)

[johnbintz]

@andreacfm, got a backtrace for that? Also, are you running HEAD or 0.9.0.rc1?

[andreacfm]

Using rc1. I have 2 issues. I need to export lang cause if I do not I get this:

[jasmine-headless-webkit] /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/vendor/assets/javascripts/beautify-html.js has a invalid US-ASCII byte sequence (Sprockets::EncodingError)
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/sprockets-2.1.2/lib/sprockets/utils.rb:18:in `block in read_unicode'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/sprockets-2.1.2/lib/sprockets/utils.rb:12:in `tap'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/sprockets-2.1.2/lib/sprockets/utils.rb:12:in `read_unicode'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/sprockets-2.1.2/lib/sprockets/context.rb:171:in `evaluate'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/sprockets-2.1.2/lib/sprockets/processed_asset.rb:12:in `initialize'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/sprockets-2.1.2/lib/sprockets/base.rb:241:in `new'

After exporting lang the full stack error is:

[jasmine-headless-webkit] Not an asset: nil (Jasmine::Headless::InvalidUniqueAsset)
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/unique_asset_list.rb:4:in `<<'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/files_list.rb:267:in `add_path'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/files_list.rb:113:in `block in load_initial_assets'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/files_list.rb:112:in `each'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/files_list.rb:112:in `load_initial_assets'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/files_list.rb:98:in `initialize'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/runner.rb:235:in `new'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/runner.rb:235:in `files_list'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/template_writer.rb:21:in `write'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/runner.rb:188:in `run'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/lib/jasmine/headless/command_line.rb:18:in `run!'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/gems/jasmine-headless-webkit-0.9.0.rc1/bin/jasmine-headless-webkit:10:in `<top (required)>'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/bin/jasmine-headless-webkit:19:in `load'
  /Users/andrea/.rvm/gems/ruby-1.9.2-p180-patched/bin/jasmine-headless-webkit:19:in `<main>'

[anthonygreen]

Running

jasmine-headless-webkit -f JUnit:junit.xml 

I got the error

[jasmine-headless-webkit] invalid option -- f (GetoptLong::InvalidOption)

I assumed this was because I'd used Gemfile and Bundler to install.

Running

bundle exec jasmine-headless-webkit -f JUnit:junit.xml

I get a different error

couldn't find file 'jasmine-reporters/src/jasmine.junit_reporter.js'

---

OK I found the issue was with the git submodule dependency on jasmine-reporters My fix was to cd into the gem and do:

git submodule init
git submodule update

to pull in the submodule

Would be nice given the general use of Bundler to manage gem dependencies a better solution than mine could be found.

[ElChapitan]

@anthonygreen I ran into similar problems, and found that most if it had to do with old versions of jasmine-headless-webkit sitting around that I hadn't noticed before. The fact that you get very different results from bundler and your regular command line make me think you're seeing the same thing.

[mfoo]

I've just been round this same loop with the same issue as @anthonygreen. Could this information be added to http://johnbintz.github.com/jasmine-headless-webkit/ ?