covr::coveralls() from github actions?

[dblodgett-usgs]

I consistently get:

 Error in covr::coveralls() : 
  Failed to upload coverage data. Reply by Coveralls: Couldn't find a repository matching this job.

Has anyone been able to get coveralls() working from github actions with this function? Possibly a way to pass parameters to make it work without any code modification? I've not been able to find any examples.

Scanning the code for covr::coveralls(), it appears that it is pretty specific to a travis-ci and possibly jenkins based CI environment. I'm having no luck getting it working and really like to maintain my history in coveralls if that's possible.

It seems like there are two options -- 1) add github-actions logic to covr::coveralls() or 2) write a file that would be compatible with the coveralls action.

I think #442 is more or less calling for (2)?

I'm not versed in LCOV -- how far is the output of this: from being an LCOV file? https://github.com/r-lib/covr/blob/master/R/coveralls.R#L43

[jimhester]

Are you supplying an explicit repo_token and service_name to covr::coveralls()?

[dblodgett-usgs]

Tried repo_token, but it wasn't clear what the service_name should be. Am I missing some documentation on that?

[dblodgett-usgs]

Tested with service_name = github-actions which is what appears in some googling .. no dice. https://github.com/dblodgett-usgs/nhdplusTools/runs/1455018761

[jefferis]

@dblodgett-usgs did you get anywhere with this in the end? I am in the same boat with a coveralls history. Thanks!

[dblodgett-usgs]

No. I just switched to codecov and moved on. https://github.com/USGS-R/nhdplusTools/blob/master/.github/workflows/R-CMD-check.yaml#L76

[jefferis]

OK! Thanks for confirming.

[adigherman]

I got this working (finally).

Workflow steps:

• Enable Repo on Coveralls - will try to enable the repo if it's not already done.
• Get Coveralls Repo Token - will get the repo_token for the package repo
• Generate and Upload R Package Coverage - generate and uploade coverage to Coveralls.

IMPORTANT: Use service_name with either drone, jenkins or travis-pro as all 3 of them will call the same covr:::jenkins_git_info() function. If service name is different, the git[] JSON element will be empty and result in a malformed Coveralls API POST request.

      - name: Enable Repo on Coveralls
        id: enable_coveralls
        run: |
          curl -s -H "Accept: application/json" \
            -H "Content-Type: application/json" \
            -H "Authorization: token $COVERALLS_TOKEN" \
            -d '{"repo": {"service": "github","name": "'$GITHUB_REPOSITORY'"}}' \
            -X POST https://coveralls.io/api/repos
     
      - name: Get Coveralls Repo Token
        id: get_coverall_repo_token
        run: |
          echo "COVERALLS_REPO_TOKEN=$(\
            curl -s -H "Accept: application/json" \
            -H "Content-Type: application/json" \
            -H "Authorization: token $COVERALLS_TOKEN" \
            https://coveralls.io/api/repos/github/$GITHUB_REPOSITORY \
            | jq '.token')" >> $GITHUB_ENV

      - name: Generate and Upload R Package Coverage
        run: covr::coveralls(service_name='drone', repo_token=${{ env.COVERALLS_REPO_TOKEN }})
        shell: Rscript {0}

You will need to set up an encrypted secret for the repo (https://docs.github.com/en/actions/reference/encrypted-secrets), and add that to the workflow environment: (orgs can share it for all repos)

    env:
      COVERALLS_TOKEN: ${{ secrets.COVERALLS_TOKEN }}

[helgasoft]

@adigherman, WOW, hats off - master stroke! 👑 The only thing I struggled with was the token. Coveralls have two types of tokens - api and repo. What worked for me was the API token.

1. generated it from https://coveralls.io/account under "PERSONAL API TOKENS"
2. in GitHub > Secrets > created a New repository secret called 'COVERALLS_API'
3. placed COVERALLS_API in the YAML env section like so:

    env:
      GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
      COVERALLS_TOKEN: ${{ secrets.COVERALLS_API }}

4. In YAML I added your code just as it is.

Thank you for the time spent on this tip!

[daattali]

Another vote to having some documentation about how to use coveralls in github actions. This package explicitly mentions that it works for codecov AND coveralls, and the README says to use usethis::use_github_action("test-coverage") but it doesn't say anything about the fact that this line will only use codecov and how it can be modified to use coveralls instead.

[daattali]

I can confirm that in order to get coveralls working in a GitHub Action, the following steps worked for me (in a private repository, so this should work for anyone):

• On coveralls website, go to your repository page to obtain the repository token. As of May 2023, that page is https://coveralls.io/github/<gh user>/<gh repo>

• On GitHub, in your repository, go to the Settings page and add a new Secret environment variable named COVERALLS_REPO_TOKEN where the value is the token from the previous step

• In your R session, call usethis::use_github_action("test-coverage")

• Call usethis::use_coverage("coveralls")

• In the resulting test_coverage.yml file, find the line that calls covr::codecov() and replace codecov with coveralls

• You'll need to add two environment variables under the env: field of this yaml file: COVERALLS_TOKEN and CI_NAME. Here is what the env: section should look like:

  env:
    GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
    COVERALLS_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
    CI_NAME: jenkins # see https://github.com/r-lib/covr/issues/457
  

  The COVERALLS_TOKEN envvar is self-explanatory, it's the token that connects this repo to coveralls. The reason we need to set CI_NAME to "jenkins" is because, as @adigherman found out, using github-actions does not work. It's probably something to do with this code https://github.com/r-lib/covr/blob/caed186fc47884d1e848c09ddb9433716bd671c6/R/coveralls.R#L65-L70

[daattali]

@helgasoft @adigherman when you used coveralls with github actions, did you get the badge to work? I'm getting "unknown" in the badge, and I suspect it potentially could be because of sending it as "jenkins" and not grabbing the right branch or something similar.

[helgasoft]

yes, badge is working. with YAML file to look at.

[daattali]

Ah you found out the secret ingredient, that I just figured out myself after looking at coveralls documentation. "CI_BRANCH" envvar is required. So you hardcoded it in, which works for us as a one-off, but {covr} really should make this work better. I'll open up an issue to resolve this so that people don't need to manipulate their environments so much.