Integration of mull with GitHub Annotations

[ligurio]

There is a feature in GitHub when CI post to it an output of static analysers or test output it annotates source code which triggered error. Example:

It would be nice to support mull in GitHub Annotations.

GitHub Annotations supported by:

• CirrusCI https://medium.com/cirruslabs/github-annotations-support-227d179cde31
• reviewdog https://github.com/reviewdog/reviewdog#supported-ci-services

[chgans]

AFAIU, for CirrusCI you just need to generate a Junit report.

I actually do this in CI, I convert clang-tidy and clazy into a fake test suite report, I use a slightly modified version of https://github.com/PSPDFKit-labs/clang-tidy-to-junit

Basically as long as mull outputs it's diagnostic the same way as clang-tidy, you can use that script. clang-tidy diagnostics are very similar (same as?) to regular clang/gcc compiler disgnostic messages

See attached screenshot for Jenkins visuals

[AlexDenisov]

I believe it's resolved via #917

[AlexDenisov]

Actually, I'm having issues seeing #917 working, so I'll reopen this one.

[daantimmer]

@AlexDenisov I too ran in to this issue and noticed it has nothing to do with mull's github annotations reporter. The generated annotations are fine. What does not work is when mull-runner is executed through ctest. Then ctest prepends each output line with a number of which test executable is executed. I created a test action to only echo the mull's reporter output with and without what cmake is doing to the output and removing the ctest output makes the github annotation report work as expected:

This does not work:

        echo "1: [info] Github Annotations:"
        echo "1: ::warning file=--redacted--cpp,line=9,col=48,endLine=9,endColumn=49::[cxx_sub_to_add] Replaced - with +"

This does work:

        echo "[info] Github Annotations:"
        echo "::warning file=--redacted--cpp,line=9,col=48,endLine=9,endColumn=49::[cxx_sub_to_add] Replaced - with +"

See:

A simple test action that can be used:

---
name: Test Annotate

on:
  push:
  pull_request:
    types: [opened, synchronize, reopened]
  workflow_dispatch:

permissions:
  contents: read

concurrency:
  group: ${{ github.ref }}-${{ github.workflow }}
  cancel-in-progress: true

jobs:
  TestAnnotate:
    runs-on: [--toyourliking--]
    steps:
    - name: Generate Warning Annotations with ctest output
      run: |
        echo "1: [info] Github Annotations:"
        echo "1: ::warning file=redacted,line=9,col=48,endLine=9,endColumn=49::[cxx_sub_to_add] Replaced - with +"
        echo "1: ::warning file=redacted,line=59,col=15,endLine=59,endColumn=17::[cxx_ge_to_gt] Replaced >= with >"
        echo "1: ::warning file=redacted,line=59,col=27,endLine=59,endColumn=29::[cxx_le_to_lt] Replaced <= with <"
    - name: Generate Warning Annotations without ctest output
      run: |
        echo "[info] Github Annotations:"
        echo "::warning file=redacted,line=9,col=48,endLine=9,endColumn=49::[cxx_sub_to_add] Replaced - with +"
        echo "::warning file=redacted,line=59,col=15,endLine=59,endColumn=17::[cxx_ge_to_gt] Replaced >= with >"
        echo "::warning file=redacted,line=59,col=27,endLine=59,endColumn=29::[cxx_le_to_lt] Replaced <= with <"

Ofc replace redacted with whatever you want in your own test repo and the same applies to the runs-on field

[AlexDenisov]

I was missing the annotations due to the relative path in the logs (../foo.cpp didn't match foo.cpp). Now fixed https://github.com/mull-project/mull/pull/1028