Please use not bot user but GitHub Apps to comment coverage to PR

[Warashi]

You can use GitHub apps to comment coverage to PR. This allows user not to invite bot user as outside collaborator.

[afinetooth]

Hi, @Warashi. Thanks for the input. We are aware of this and plan to make the switch to Github App-based integration later this year. It's not a simple matter for us due to the number of legacy repos that are in set-and-forget mode, so we will need to support both for some time and provide an easy transition step, until we fully deprecate the current method and migrate everyone.

In the meantime, the Coveralls Github Action is an available integration that relies solely on Github App-based access.

If you're using Github Actions for CI and want to transition to that integration, let us know and we'll help you port over.

[Warashi]

Thanks for reply. My project uses Go, so I'm using goveralls with actions-goveralls. Is there the way to use suggested action with goveralls?

P.S. very thanks to make amazingly useful service.

[afinetooth]

Hi, @Warashi. Upon reading these usage instructions for the Github Action version of mattn/goveralls, I feel confident that actions-goveralls uses Github App-based access to your repo.

However, Coveralls is not specifically aware of that integration, so it's not aware of the option to send PR Comments via Github App. Therefore it will send PR comments the normal way, which means by leveraging the Coveralls bot user, who needs to be added to your repo as described here. (Which I assume you've done.)

Please note that the Coveralls bot is used only by coveralls.io, only for the purpose of leaving PR comments, and is secured via 2FA that only our CTO can auth through. But if you have security concerns you can avoid giving the bot write permission to your repo by following these instructions.

[Warashi]

Hi, @afinetooth. Thanks for checking about goveralls.

Please could I explain why I want you to use GitHub Apps for sending comments. My company forces SAML authentication for organization members, so I cannot invite bot user as org-wide outside collaborator. Because this reason, I must invite bot user each repository one-by-one. This is not so enjoyable experience for me. I don't concern about security about using bot user. I trust your security.

[Warashi]

I found gcov2lcov-action which is action to convert Go coverage file to lcov format. This allows me to use official the Coveralls Github Action, so this is one of solution for me. Thanks for help me!

[afinetooth]

Hi @Warashi, I see. Thank you for the explanation, that's a very valid use case. I have submitted a feature change request to always send pull request comments via Github App if the repo has an associated Github App.

Glad you found gcov2lcov-action and that this is a solution for you! Good find!

[afinetooth]

@Warashi I will close this issue if that's OK. Just re-open if necessary.

[joebowbeer]

We are aware of this and plan to make the switch to Github App-based integration later this year.

@afinetooth Is there an open RFE for the planned switch to GitHub App? I would like to know more about what is planned.

[afinetooth]

Hi @joebowbeer. While there is an RFE in our backlog, it's not currently scheduled for any upcoming sprints. The main reason is that there is a workaround for Github organizations that don't wish to give the Coveralls Bot write access to repos.

That request lives across a number of cards. Is there something specific I can tell you about it?

[Warashi]

@afinetooth I cannot re-open this issue, but I want to ask how to use the Coveralls Action.

I remove @coveralls from the repository outside collaborators, then coverage comments are not left at my PRs. I'm using Coveralls GitHub Action. When I invite back @coveralls as the outside collaborator, then coverage comments are left successfully. How can I use Coveralls GitHub Action and coverage comments without an outside collaborator?

[afinetooth]

@Warashi please let me look into this further. I may have been mistaken that, because the Coveralls Github Action uses a Github App, it leverages that app for PR Comments. I was under the impression that is how it works, but I don't see the evidence looking through the source code right now, so I will need to get further input and verify one way or the other.

[Warashi]

@afinetooth please re-open this issue? I don't have permission to do.

[afinetooth]

@Warashi reporting back to confirm we do not leverage the Github App underlying the Coveralls Github Action to send PR comments. Instead, we currently have one method that sends PR Comments, leveraging the token of our Coveralls bot.

I found an existing card in our backlog requesting we leverage the Github app underlying a Github Action when being used by any repo / build, before we try to send via our bot user. I "plus-oned" you on that card with this issue as a reference.

FWIW, the Coveralls bot is only used for the purpose of leaving PR Comments. It's not used by any internal users at coveralls, and it is not even accessible to general employees. It is set up with 2FA that only our CTO can access. So while the situation is a risk in principle, in practice the risk should be relatively low compared to other Github accounts having access to your repo. And this only really applies if you can't use this method to avoid giving the bot write access.

[Warashi]

thanks for reporting.

FWIW, the Coveralls bot is only used for the purpose of leaving PR Comments. It's not used by any internal users at coveralls, and it is not even accessible to general employees. It is set up with 2FA that only our CTO can access. So while the situation is a risk in principle, in practice the risk should be relatively low compared to other Github accounts having access to your repo. And this only really applies if you can't use this method to avoid giving the bot write access.

As I wrote before, I trust your security. We cannot invite bot user as org-wide outside collaborator because of SAML Auth requirements. So We have to invite bot user each repository by hand. I don't feel like do this each time when I create repository.

[afinetooth]

hi @Warashi,

We cannot invite bot user as org-wide outside collaborator because of SAML Auth requirements.

Got it. I understand.

So We have to invite bot user each repository by hand. I don't feel like do this each time when I create repository.

I understand. I wouldn't either.

I plus-one'd you on the card in our backlog for the Github App based solution. Thanks.

[fubhy]

This is also a show stopper for us as we can't, by policy, add an arbitrary user (bot or not) to our private repositories. Granting read access only isn't a solution for us either. Not because of the usability concerns (adding it to each repository manually) but again simply due to our strict policies. Trusting that only your CTO has access to the bot isn't really reassuring either and even if I wanted to trust that statement it still wouldn't be acceptable for us to grant access to an external bot. I am sure other teams bound by ISO certification and company policy constraints are in the same boat.

I am really looking forward to seeing a solution for this via Github Apps. I hope there's any way this can be prioritized :)

[afinetooth]

Hi @fubhy. Thanks for the feedback. Fully understand your position. I would like to reassure you that it's definitely a high priority for us. I will feed back as soon as I can say anything more meaningful than that.