dependabot-core icon indicating copy to clipboard operation
dependabot-core copied to clipboard

Dependabot PR's actions lost access to secrets

Open ShockwaveNN opened this issue 1 year ago • 26 comments

I think something changed in the logic of GitHub actions triggered via dependabot update, around a week ago, but I didn't find any mention in any GitHub changelog

So, my situation is:

I've got several projects of RubyGems, which triggers GitHub Actions with specs on each PR. Those actions require access to secrets, and fail without secrets

Before a week ago if I got Dependabot PRs - all Actions failed, but if I retrigger them manually with my usual Github account via web interface - after retrigger they became aware of secrets and become green

But something happened last week and now the only way to trigger Action with access to secrets - is to checkout this branch locally, making git commit --amend so GitHub think I'm an author and after that Actions are green

Could you point out in the direction if I'm doing something wrong or why it was changed a week or two ago?

Package ecosystem rubygems and bundler Package manager version bundler 2.3.19 Language version ruby 3.1 dependabot.yml content

version: 2
updates:
  - package-ecosystem: bundler
    directory: "/"
    schedule:
      interval: daily
      time: "08:00"
      timezone: "Europe/Moscow"
    open-pull-requests-limit: 10
    allow:
      - dependency-type: direct
      - dependency-type: indirect

ShockwaveNN avatar Aug 02 '22 07:08 ShockwaveNN

Thanks for reporting this, I'm not aware of anything having changed, but I will check with the Actions team! Could you share your actions workflow file as well maybe?

jurre avatar Aug 02 '22 09:08 jurre

I can share the whole repo, one of my several

https://github.com/ONLYOFFICE-QA/onlyoffice_s3_wrapper/blob/master/.github/workflows/ruby.yml

I'm surprise that I'm only one reporting this, I've checked issues at Dependabot and Github Actions daily and today I'm finally gave up and made my own report )

I got a little chance that I'm doing something wrong and maybe you can help me with that, or maybe it's really an issue

ShockwaveNN avatar Aug 02 '22 09:08 ShockwaveNN

          S3_KEY: ${{ secrets.S3_KEY }}
          S3_PRIVATE_KEY: ${{ secrets.S3_PRIVATE_KEY }}

Do you have those secrets set up as both Dependabot secrets and Actions secrets? It's curious to me that it used to work and stopped working, but Dependabot triggered actions have access to Dependabot secrets but not Actions ones by default, so exposing them as Dependabot Secrets could be a potential fix if they're not.

jurre avatar Aug 02 '22 09:08 jurre

We have this same issue at the moment. Actions triggered by dependabot would fail as it did not have access to secrets, and then someone from the team could retrigger them and they would now pass (As the user triggering the action to rerun has access to those secrets).

Secrets are not configured as dependabot secrets, However we would prefer not to add these secrets to dependebot and prefer retriggering them as a user after the changes have been reviewed.

mkrakowitzer avatar Aug 02 '22 10:08 mkrakowitzer

@jurre

Do you have those secrets set up as both Dependabot secrets and Actions secrets?

No, where was never Dependabot specific secrets configures, as I mention earlier I've triggered task manually

Frankly this is first time I'm aware that specific secrets for Dependabot are exists )

ShockwaveNN avatar Aug 02 '22 10:08 ShockwaveNN

@mkrakowitzer

Actions triggered by dependabot would fail as it did not have access to secrets, and then someone from the team could retrigger them and they would now pass (As the user triggering the action to rerun has access to those secrets).

This is the point. For me it was like this everytime as I understand is correct behavior But right now even if I trigger action manually via my personal account - dependabot actions still have no access to secretd

ShockwaveNN avatar Aug 02 '22 10:08 ShockwaveNN

That's surprising, manually triggered workflows should have access to regular Actions secrets. I'm following up with the Actions team as it's not something we can fix or even really investigate on our end. Will keep you posted as I hear more from them.

Edit: turns out this has not been true for a while

jurre avatar Aug 02 '22 10:08 jurre

@jurre Ok thanks, I think tomorrow morning there will be some new PRs from dependabot and I'll leave this open for you to investigate

ShockwaveNN avatar Aug 02 '22 10:08 ShockwaveNN

So, the Actions team have gotten back to me with:

re-run has used the original actor at least in that case of dependabot.

So it does seem like this is by design, and the suggested solution is to add the secrets as Dependabot secrets. I'm trying to find if there is any communication that we've sent out on this, and looking to add it to our documentation.

jurre avatar Aug 02 '22 11:08 jurre

What do we do in the case where we do not want a third party (dependabot) to have access to these secrets?

mkrakowitzer avatar Aug 02 '22 11:08 mkrakowitzer

I did manage to find this mentioned in our docs, for what it's worth: https://docs.github.com/en/code-security/dependabot/working-with-dependabot/automating-dependabot-with-github-actions#manually-re-running-a-workflow

What do we do in the case where we do not want a third party (dependabot) to have access to these secrets?

Dependabot is not a third party, we're a native to and a part of GitHub, the way we store secrets is shared with the way Actions does it. The code in dependabot-core does not have direct access to these secrets, instead requests are routed through a short-lived and job-specific proxy where the credentials are appended to requests if they match a configuration in your dependabot private registry config.

If you still don't want to configure this, right now the only way I can think of to make this work is something fairly contrived, like to have some sort of action that monitors PRs opened by Dependabot, grabs it's contents and re-opens a new PR under a different user. I would not recommend this though.

jurre avatar Aug 02 '22 11:08 jurre

If the GitHub team needs more information, I wrote up an issue about the previous behavior and the new behavior: https://github.com/web-platform-tests/wpt.fyi/issues/2928

jcscottiii avatar Aug 02 '22 19:08 jcscottiii

And an example that I've promised

So this is actions run which was created by dependabot, I've not touched anything in it It shows empty secrets: image

And this one which I've retriggered manually by main usual main account (this one) - same problem with empty keys

Am I right that I need to redone my configuration to use Dependabot secrets? It's no problem for me

But I still do not understand that the Actions team has confirmed that something changed in dependabot configs a week or two ago. Or does no one know why this happened?

Scenario with manual retrigger of dependabot update 100% worked for me several weeks ago daily, I cannot we wrong about this

ShockwaveNN avatar Aug 03 '22 07:08 ShockwaveNN

If you add them to dependabot secrets should work. However, there are additional security concerns you should be aware of. @jcscottiii documents it in his comments above. https://github.com/web-platform-tests/wpt.fyi/issues/2928.

Would also love to know why this has changed. We prefer the old method, so that we can verify the dependabot changes before allowing the workflows to run.

mkrakowitzer avatar Aug 03 '22 07:08 mkrakowitzer

So, if I understand things correctly from the Actions team, back in the day re-runs would use the permissions of the actor performing the re-run, but a while ago this was changed to reuse the permissions of the original actor, but there might have been a regression for a while where this didn't work as intended. I will point the right folks to this thread, and see if I can learn more about why this behavior is the way it is.

jurre avatar Aug 03 '22 07:08 jurre

but a while ago this was changed to reuse the permissions of the original actor, but there might have been a regression for a while where this didn't work as intended

Yeah, it would great if it will be mention in Github Changelog or somewhere

ShockwaveNN avatar Aug 03 '22 07:08 ShockwaveNN

@jurre we are suffering from the same issue. We have a lot (and I mean a lot) of secrets that we do not give access to bots (dependabot, renovate) and our workflow for all PRs opened by those, was (up until recently) to re-run their build by someone from the team.

back in the day re-runs would use the permissions of the actor performing the re-run, but a while ago this was changed to reuse the permissions of the original actor,

I do not understand this change to be honest, the whole point of a re-run was to do it with the permissions of the person triggering run that has the rights to access secrets and whatnot.

Is there an explanation from actions team, why the change was made or is it a regression?

gustaff-weldon avatar Aug 08 '22 09:08 gustaff-weldon

A possible way/suggestion to unbreak the break; would be perhaps to either revert the "fix" or add

  • "rerun as original actor"
  • "rerun as user"

mirageglobe avatar Aug 09 '22 15:08 mirageglobe

@jurre @mischa0083 Any new findings on why this has changed?

knjoroge avatar Aug 17 '22 09:08 knjoroge

This change was announced in this blog post

I think the main intent is to create more consistent builds when performing re-runs.

pavera avatar Aug 17 '22 17:08 pavera

Ok if this was intentional and announced I think we can close this issue, personally I've adapted all my Actions to call dependabot secrets, so this is not a problem anymore

Any thoughts?

ShockwaveNN avatar Aug 18 '22 07:08 ShockwaveNN

@pavera this behaviour breaks people's workflows, I think "rerun as user" might make sense.

I do not see a solution here at the moment, if one does not want give dependabot immediate access to "dependabot" secrets. Even then it does not solve the "renovate" bot issue.

@ShockwaveNN I do not think giving access to secrets to bots is a safe thing. When bots access to secrets was taken away it was due to security concerns, do you think those do not exist anymore?

gustaff-weldon avatar Aug 18 '22 07:08 gustaff-weldon

I do not think giving access to secrets to bots is a safe thing

Personally I don't think it's some kind of bot, it's a part of Github for quite a long time

It's even have setting in repo settings, not in some kind Third party bots secret share page

image image

So if I share my keys with Github in form of Github Actions secrets, why I sholdn't share secrets with Github in form of Dependabot secrets?

ShockwaveNN avatar Aug 18 '22 07:08 ShockwaveNN

@ShockwaveNN it's not the point whether it is a dependabot or renovate bot here, the point is what can run on PRs created by such bots if you allow them.

I suggest you read the comment I linked to above. Automatically running workflows on PRs created by bots is a security threat, due to possible malicious dependency updates.

Why was this change made? This change was made in response to reported vulnerabilities in these types of workflows, and we felt that it critical to ensure our developers were protected before they were publicly disclosed.

Specifically, a malicious dependency could execute code to exfiltrate secrets or perform something other malicious action in the repo (deleting files, etc.).

By making secrets unavailable and tokens read only, this prevents a compromised dependency from being able to exfiltrate those secrets or perform any malicious write actions.

This is the reason why many people, including us, relies on re-running the workflow as project team user after verifying the content of the PR.

gustaff-weldon avatar Aug 18 '22 07:08 gustaff-weldon

@gustaff-weldon Ok, fair enought in that case I will not close this issue as author, but for me it's not a trouble any more

ShockwaveNN avatar Aug 18 '22 07:08 ShockwaveNN

👋 @ShockwaveNN @gustaff-weldon (and everyone else chiming in here) Thanks for bringing this up and sharing your concerns 😄

🔎 As I read this thread, I notice that the concern is more specific to how GitHub runs Dependabot than the logic that powers dependency updates and pull requests. Unfortunately, changes that happen in this repository won't affect the way Dependabot interacts with secrets. I think your concerns are 100% valid and deserving of the right audience.

GitHub has a feedback forum that I think is better suited for this topic. I'm going to open a discussion under the code-security topic and link back to this issue. The new discussion will have the Actions and Packages and Dependabot labels, so 🤞 we can get your concerns heard by the respective teams.

Once the discussion is kicked off, I'll make sure to come back and update this comment with a link to it. I'd love it if this conversation could move to that forum and we could come to a resolution that everyone involved at least understands.


Edit: the discussion has been started 🎉 Please feel free to join in 😄

landongrindheim avatar Aug 23 '22 20:08 landongrindheim

👋 Since this is a service-level concern, which is in contention with another service (GitHub Actions), I'm going to close this issue. Changes in this repository will not affect what happens on GitHub.com. If you'd like to weigh in further, please do so in the previously-opened discussion 😄

landongrindheim avatar Mar 31 '23 12:03 landongrindheim