Motivation

Add code coverage to make sure good quality of PRs. Although unit tests run in different runners, by using Codecov the coverage XML reports can be aggregated automately.

We can add the coverage to take a look first and then adjust the rule of coverage.

Verifying this change

[x] Make sure that the change passes the CI checks.

Documentation

Check the box below or label this PR directly.

Need to update docs?

[ ] doc-required (Your PR needs to update docs and you will update later)
[x] doc-not-needed (Please explain why)
[ ] doc (Your PR contains doc changes)
[ ] doc-complete (Docs have been already added)

Matching PR in forked repository

PR in forked repository: https://github.com/yaalsn/pulsar/pull/7

Sep 01 '22 01:09 yaalsn

https://github.com/codecov/sourcegraph-codecov/blob/master/README.md This Chrome ext can show code coverage information from Codecov on GitHub directly which means no need to jump to Codecov to take look.

Sep 01 '22 02:09 yaalsn

@yaalsn can you share the report of Pulsar right now? I'm afraid that we just add one more workflow while no one cares about it especially when the result is overwhelming all consumers.

Sep 01 '22 02:09 tisonkun

@tisonkun Thanks for your reminder.

https://app.codecov.io/gh/apache/pulsar/tree/add-code-coverage This is this PR's coverage report, and Codecov will comment a message to this PR's body after the CI uploads the report.

Sep 01 '22 02:09 yaalsn

I added some configuration to codecov.yml in my repo's pulsar because of the apache/pulsar's runner resources lacking. This is the result: https://github.com/yaalsn/pulsar/pull/11

We can get the uncovered part from Files Changed
The coverage and impacted files general information in comment

Status check in action check, but it doesn't block the merge even if the coverage down

Sep 15 '22 00:09 yaalsn

@tisonkun @nicoloboschi Could you take a look at the coverage data again? It looks good to me, what do you think?

Sep 15 '22 02:09 yaalsn

@yaalsn interesting. The result seems not quite correct or useful.

Sep 15 '22 05:09 tisonkun

And it seems quite noisy. Perhaps some settings are incorrect.

Sep 15 '22 05:09 tisonkun

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@2b9ffac). Click here to learn what that means. The diff coverage is n/a.

@@            Coverage Diff            @@
##             master   #17382   +/-   ##
=========================================
  Coverage          ?   45.92%           
  Complexity        ?    17525           
=========================================
  Files             ?     1569           
  Lines             ?   127638           
  Branches          ?    14038           
=========================================
  Hits              ?    58614           
  Misses            ?    63028           
  Partials          ?     5996

Flag	Coverage Δ
unittests	`45.92% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Sep 15 '22 05:09 codecov-commenter

@yaalsn interesting. The result seems not quite correct or useful.

@tisonkun Because there are some flaky tests, and if the job run success, the result will be different. If we want to turn off the email notification, we need to disable the comment from codecov too.

Another way is to change the behaviour to default, and it will send a email at the first time. (but it is not the final result)

Sep 15 '22 08:09 yaalsn