support annotations instead of pr comments / reviews

[valentijnscholten]

Great tooling, but personally I find it more useful to have the violations reported as annotations on the "Files Changed" tab. Having them as comments in the PR itself (or a review) clutters the PR.

Example:

[grantmcconnaughey]

Agreed! Lintly supports a flag called —use-checks, which uses GitHubs Checks API. It works when running Lintly from GitHub Actions. That should get the result you’re looking for.

[valentijnscholten]

Ah oke, I didn't know that was tied to the Checks API. I will try with a modified version of the lintly-flake8 action.

[grantmcconnaughey]

Do you think it would be a useful default to use the Checks API if Lintly runs from GitHub Actions? With the ability to override if someone prefers PR reviews.

[valentijnscholten]

For me that would be a better default yes

[joshuacwnewton]

I've tried enabling --use-checks as recommended above, but by default, it puts the annotations in a detached commit with the following message:

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Here is an example: https://github.com/neuropoly/spinalcordtoolbox/commit/353d15aaa0094a6d4fc135a86b49df0092b6e122. As a result, the annotations are not visible in the PR files page.

My GH Actions workflow config is the following:

name: Python Linting (lintly + flake8)

on: [pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: 3.8
    - name: Install dependencies
      run: pip install flake8 lintly==0.5.0
    - name: Lint with flake8
      run: flake8 | lintly --use-checks --fail-on new --exit-zero --no-post-status
      env:
        LINTLY_API_KEY: ${{ secrets.GITHUB_TOKEN }}

Solution

I was able to fix this by manually specifying a different commit SHA. I learned which commit SHA to use by looking at the source code for flake8-your-pr. Now, my workflow config looks like this:

      run: flake8 | lintly --use-checks --fail-on new --exit-zero --no-post-status --commit-sha ${{ github.event.pull_request.head.sha }}

After this change, the annotation properly shows up in correct commit, and in the Files dialog: https://github.com/neuropoly/spinalcordtoolbox/pull/3213/files

I'm mentioning this here for anyone reading in the future, or in the hopes that this fix could be applied upstream.

(Thank you for developing this lovely tool!)