Add coverage report badge using Codecov

[rubenochiavone]

Description

Add code coverage report to whole project (not only for each sub-project) and code coverage badge to README.md.

Question

• Code coverage badge relies on https://codecov.io/. Is using https://codecov.io/ OK for solidus? If so, it would be great if someone could provide a CODECOV_TOKEN to be added in this PR.

Fixes #3048

Checklist:

• [x] I have followed Pull Request guidelines
• [x] I have added a detailed description into each commit message

[kennyadsl]

@rubenochiavone I've set up a CODECOV_TOKEN env variable in CircleCI that should be visible in all builds but hidden publicly. Let me know if that works

[rubenochiavone]

Hi there :wave:

It's been a while since I last worked on this PR, but I think it is OK for being reviewed now.

Things worth commenting on:

• Merged coverage results both locally and on CI
• Added COVERAGE_DIR environment variable. This variable refers to the directory that will accumulate coverage results between each sub-project test execution and then be used to calculate complete coverage result
• Set 1 hour timeout to calculate merged coverage locally, see https://github.com/solidusio/solidus/pull/3136/files#diff-0c169400cff5f1673e3dffe059269a26c18ca969c470b93ff9b94587b1b56562R9 and https://www.rubydoc.info/gems/simplecov/SimpleCov%2FConfiguration:merge_timeout. This fixed value could lead to not accurate coverage results if the call COVERAGE=true bin/build takes more than 1 hour. This cannot be tuned right now, but could be done using an environment variable. Any thoughts on that?

Current coverage report:

On #3048 I mentioned that would be interesting to have a report once a PR is created. So to check that I created a test PR to assess coverage validation during PRs and it is working on my fork: https://github.com/rubenochiavone/solidus/pull/1. Installing codecov app on the project already does it - https://docs.codecov.io/docs/team-bot.

Sorry for accidentally creating PR #3848 :grimacing:

[kennyadsl]

Hey @rubenochiavone, thanks for resuming this work! Do you need anything from me to be changed in the CI (like adding secrets or stuff like that)?

[rubenochiavone]

Hi @kennyadsl, the token you configured was enough. Check out the coverage report here: https://codecov.io/github/rubenochiavone/solidus/commit/b342cd9123a4d831087f034de0d66acc1f805156. I think that this PR is ready for review. Please let me know if this PR needs more work.

[waiting-for-dev]

Hey @rubenochiavone, I'm sorry it's been a while since we last moved it forward. Would you like to rebase it?

[rubenochiavone]

Sure. :smiley:

[rubenochiavone]

Hey @waiting-for-dev, besides the rebase, I had to rework the CI script to upload coverage data to codecov.io due to the deprecation of the codecov gem - see https://github.com/codecov/codecov-ruby#readme. The new codecov script also required a different coverage data format, thus I had to change it while generating results in CI.

Current coverage:

Please let me know what you think about it.

[waiting-for-dev]

Hey @rubenochiavone, thanks for your continuous work on it 🙌 It looks like there're still some conflicts to resolve.