Calculable fields should be server-side calculated

[pbrisbin]

:wave: hey CC, how're things!

I'm writing a parser-formatter that takes coverage information about Haskell tests in its native format and outputs it as the JSON payload expected by cc-test-reporter.

I'm working against your schema.json, and I'm specifically only including data marked as required in said schema:

• Git information:

  • branch
  • commit sha
  • commit timestamp

• Source files

  • path
  • blob id
  • coverage array

I'm producing valid payloads that are accepted by the reporter, and the line coverage is accurately displayed on the site:

https://codeclimate.com/github/pbrisbin/hs-shellwords/src/ShellWords.hs/source

Here's my most recent payload:

{
  "git": {
    "branch": "master",
    "committed_at": 1523657633,
    "head": "151184831f27726c5450255726d696f044694bc1"
  },
  "source_files": [
    {
      "name": "src/ShellWords.hs",
      "blob_id": "9a6c61963caa274c5f5d9321600096f53ffc3080",
      "coverage": "[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,1,null,null,1,null,null,1,null,null,null,1,1,10,null,null,null,1,1,1,null,null,null,1,1,1,null,null,null,1,1,1,null,null,null,1,null,null,11,null,null,1,null,null,1]"
    }
  ]
}

What's going wrong: coverage for the repository is reported as 0%.

It seems the backend is defaulting the covered_percent values I'm omitting to 0.

What I'd love: calculate omitted calculable fields server-side.

When I saw that things like covered_percent were optional, I assumed (and I think it's reasonable to assume) that those values would be calculated by something downstream based on the source_files[].coverage arrays I am submitting.

The same could be said of other calculable fields like covered_strength and line_counts, though I can't tell in the UI if those are being defaulted to "empty" values too.

If this can't happen, or won't happen just yet, I would recommend you update your schema.json to indicate these fields are required. The system doesn't really work with such calculated fields defaulted to empty values when omitted.

What I'd really love is if any calculable fields were removed from the schema entirely and just always calculated. I think that'd be simpler on your end as well.

I'll probably just start working on doing the calculations client-side to unblock myself, but I'm interested what you think of my suggestion.

As a workaround, I thought I could write a small converter from the Haskell format into gcov, which is already reportable, but then I ran into #328.

[ale7714]

👋 @pbrisbin great suggestions, thank you! We are planning to update our server-side process to reduce some of the redundant information that's currently required. But in the meantime, I'll follow up this comment by updating the schema.json file to reduce confusion.