Update repo for Crystal 1+ and document how to run the tests; maybe add an example shard that uses it

[drhuffman12]

I tried forking and updating/fixing this repo, but am having issues. What am I missing?

Please add to the doc's re running the tests (and update the repo to work with Crystal 1+). I think the coverage process is not counting the lines that the specs are actually covering.

• Update the repo to work with Crystal 1+. (See also https://github.com/anykeyh/crystal-coverage/issues/15.)
• Update the documentation.
  • How to run the tests of this repo?
  • How to use the coverage script on another repo?
  • What the coverage report should look like (e.g.: screenshots)?
• Fix the coverage script so that the lines covered by the tests are getting counted in the coverage report. (Please forgive me if I'm misunderstanding how to run the coverage script.)
• Maybe also add an example shard that uses this tool.

For details, see below...

I see spec/run.sh which includes:

#!/bin/sh
crystal src/coverage/cli.cr -- spec/template.cr --use-require="./src/coverage/runtime" -p

I had errors when running that. First, ameba needed to be switched from veelenga/ameba to crystal-ameba/ameba.

The next error was:

$ spec/run.sh
Showing last frame. Use --error-trace for full trace.

In src/coverage/inject/cli.cr:11:20

 11 | OptionParser.parse! do |parser|
                   ^-----
Error: undefined method 'parse!' for OptionParser.class

I chased down that issue and some others in a fork/branch (https://github.com/drhuffman12/crystal-coverage/pull/1) and I seemed to get the scripts to work eventially. I added an example shard (https://github.com/drhuffman12/crystal-coverage-example/pull/1) using my fork. I had mixed success.

In my example shard, I tried running the coverage script via:

• lib/coverage/bin/crystal-coverage src/**/*.cr
  • The generated web pages just found one of the src files, possibly just the 'first' found?
• lib/coverage/bin/crystal-coverage spec/**/*_spec.cr
  • The generated web pages found all 'src' and 'spec' files. Should we just show the 'src' files in the gen'd web page?

It looks like the coverage calculations/process is broken; it runs and generates the web pages but no tests seem to get logged as line coverages. Maybe I'm doing something wrong? In my example shard, I put in two duplicate classes (only diff by class name) and added more tests for one class than the other. However, the coverage for both is identical and the method that I added a test for isn't showing any code coverage:

Tests added for some Bar methods...

require "./../spec_helper"

describe Crystal::Coverage::Example::Bar do
  describe "#initialize" do
    it "creates a Bar" do
      var1 = "1 string"
      var2 = 2
      var3 = 3
      bar = Crystal::Coverage::Example::Bar.new(var1: var1, var2: var2)
      bar.should_not be_nil
    end
    context "initializes" do
      it "@var1" do
        var1 = "1 string"
        var2 = 2
        var3 = 3
        bar = Crystal::Coverage::Example::Bar.new(var1: var1, var2: var2)
        bar.var1.should eq(var1)
      end
      it "@var2" do
        var1 = "1 string"
        var2 = 2
        var3 = 3
        bar = Crystal::Coverage::Example::Bar.new(var1: var1, var2: var2)
        bar.var2.should eq(var2)
      end
    end
  end

  describe "#baz" do
    it "creates a Bar" do
      var1 = "1 string"
      var2 = 2
      var3 = 3
      bar = Crystal::Coverage::Example::Bar.new(var1: var1, var2: var2)
      bar.var2.should eq(var2)
      bar.baz
      bar.var2.should eq(var2 * 2)
    end
  end
end

... But, these test's don't seem to register with the coverage script:

[drhuffman12]

See also chat in forum: https://forum.crystal-lang.org/t/code-coverage-shard-for-crystal/3380/2

[lbguilherme]

Hi @drhuffman12!

I opened a PR that will fix this. Please take a look: https://github.com/anykeyh/crystal-coverage/pull/18

[drhuffman12]

@lbguilherme , I might need some help with this. Maybe beef up the doc's a bit more or point me to the right doc location?

I tried the following:

• mkdir $SOME_PATH && $SOME_PATH
• git clone https://github.com/lbguilherme/crystal-coverage
• cd crystal-coverage
• shards install
• shards build
• The the next commands each error re Error: can't find file 'coverage/runtime':
  • spec/run.sh
  • bin/crystal-coverage -p | crystal eval --release
  • bin/crystal-coverage
  • ./bin/crystal-coverage
  • bin/crystal-coverage spec/main.cr
  • bin/crystal-coverage src/coverage/cli.cr

[lbguilherme]

I'm using it like this:

From your project modify shard.yml to include:

development_dependencies:
  coverage:
    github: lbguilherme/crystal-coverage

Install with:

shards install

Then run specs with coverage using:

bin/crystal-coverage

[drhuffman12]

@lbguilherme , Thanks! This looks much better! :) I'd call this issue basically 'done'!

Oh, as for the "add docs" and "example shard that uses it" parts, I can move those to a separate issue(s). Feel free to pull in parts/all of my https://github.com/drhuffman12/crystal-coverage-example repo as an example.

Otherwise, feel free to close this issue. :)

I tried that in an example repo (https://github.com/drhuffman12/crystal-coverage/pull/1), without adding any code other than updating the shards. (I did have to remove the lib folder and my old shards.lock file; something to remember when debugging dependencies, but that's fine.)

I think you can mark this issue as 'fixed' and close it:

• re spec files showing up in the HTML output, this is fixed.
  • The HTML output doesn't list the *_spec.cr files anymore; just the associated spec coverage for the associated crystal src files. :)
• re covered lines and calcs, these seem fixed too; at least, my example src/crystal-coverage-example/bar.cr file seemed to show the correct coverage info.

For example, for my example repo with just a couple spec files (https://github.com/drhuffman12/crystal-coverage-example/pull/1), if I run...

• All tests, I get:

  $ bin/crystal-coverage
  ......
  Lines 64.29% covered
  
  Finished in 3.67 milliseconds
  6 examples, 0 failures, 0 errors, 0 pending
  

  ... with the following per-file coverage (HTML report snippet):

  File	Lines
  src/crystal-coverage-example/bar.cr	5 / 7 (71.43%)		
  src/crystal-coverage-example/foo.cr	4 / 7 (57.14%)		
  TOTAL:	9 / 14 (64.29%)