dex icon indicating copy to clipboard operation
dex copied to clipboard

Machine Authentication broken for Github Actions

Open Moulick opened this issue 9 months ago • 9 comments

Preflight Checklist

  • [x] I agree to follow the Code of Conduct that this project adheres to.
  • [x] I have searched the issue tracker for an issue that matches the one I want to file, without success.
  • [x] I am not looking for support or already pursued the available support channels without success.

Version

2.42.0

Storage Type

In-memory

Installation Type

Official Helm chart

Expected Behavior

Token exchange for GitHub Actions following https://dexidp.io/docs/guides/token-exchange/ should work

Actual Behavior

Does not work. Dex returns a error

dex-79d7d98fff-8qrl2 dex time=2025-03-06T13:58:35.821Z level=ERROR msg="failed to verify subject token" err="oidc: getUserInfo is required for access token exchange" request_id=e2ef51fa-84c7-4734-8ea4-ade43a95d1af

Additional Information

As per https://github.com/seankhliao/dex/blob/569e0ccbb35fb57838c9eca2846c7f182ee3b112/connector/oidc/oidc.go#L422-L425, it demands a getUserInfo be enabled for OIDC provider. But GitHub Actions token issuer does not have a user info endpoint, https://token.actions.githubusercontent.com/.well-known/openid-configuration.

Configuration

issuer: https://dex.main.odyssey.preview.nwse.cloud/dex
storage:
  type: memory
oauth2:
  grantTypes:
    - "urn:ietf:params:oauth:grant-type:token-exchange"
  responseTypes:
    - code
    - token
    - id_token
  skipApprovalScreen: true
  alwaysShowLoginScreen: false
staticClients:
  - name: Github Actions
    id: github-actions
    secret: my-secret-github-actions
    public: true

connectors:
  - type: oidc
    id: github-actions
    name: github-actions
    config:
      issuer: https://token.actions.githubusercontent.com
      scopes:
        - openid
        - groups
      userNameKey: sub
      # getUserInfo: false

Logs

dex-79d7d98fff-8qrl2 dex time=2025-03-06T13:58:35.821Z level=ERROR msg="failed to verify subject token" err="oidc: getUserInfo is required for access token exchange" request_id=e2ef51fa-84c7-4734-8ea4-ade43a95d1af

GitHub action used

name: dex-auth

on: [push]

permissions:
  id-token: write # This is required for requesting the JWT

jobs:
  job:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/github-script@v6
        id: script
        timeout-minutes: 10
        with:
          debug: true
          script: |
            const token = await core.getIDToken()
            core.setOutput('GH_TOKEN_OIDC', token)
            core.exportVariable('GH_TOKEN_OIDC', token)
            console.log(token)
      
      ### Debugging
      - id: env-dump
        run: |
          env > env.txt
       ### Debugging

      - name: env
        uses: actions/upload-artifact@v4
        with:
          name: env
          path: |
            env.txt

      - id: dex-exchange
        run: |
          # Exchange it for a dex token
          curl -v \
            https://dex.example.com/dex/token \
            --user github-actions:my-secret-github-actions \
            --data-urlencode "connector_id=github-actions" \
            --data-urlencode "grant_type=urn:ietf:params:oauth:grant-type:token-exchange" \
            --data-urlencode "scope=openid groups federated:id" \
            --data-urlencode "requested_token_type=urn:ietf:params:oauth:token-type:access_token" \
            --data-urlencode "subject_token=$GH_TOKEN_OIDC" \
            --data-urlencode "subject_token_type=urn:ietf:params:oauth:token-type:access_token"

Moulick avatar Mar 06 '25 14:03 Moulick

Can confirm, just spent several hours banging my head against this to no avail. Finally found this issue.

Makeshift avatar Apr 10 '25 15:04 Makeshift

Had this same issue and i figured out switching token type to id_token worked: subject_token_type=urn:ietf:params:oauth:token-type:id_token

phamviet avatar May 13 '25 12:05 phamviet

@phamviet funny, I'm debugging this atm too. Scary seeing "now" instead of a timestamp

Setting subject_token_type=urn:ietf:params:oauth:token-type:id_token as you suggested did fix it for me too! (and leave requested_token_type=urn:ietf:params:oauth:token-type:access_token)

One thing Dex does is it encodes the sub as base64-encoded protobuf message IDTokenSubject, as is explained in this ticket: #1719. Dex also sets federated_claims.user_id to the original sub, which servers should make use of.

Personally I've been trying to get GitHub Actions auth working for ArgoCD. ArgoCD does not make use of federated_claims.user_id, but instead it uses the sub. (This changed in ArgoCD v3: https://github.com/argoproj/argo-cd/pull/20683, however I'm still stuck at ArgoCD v2 for the time being)

So I was also stuck for a little while getting ArgoCD auth working, but eventually I figured out that I had to do this:

# Helm values for the argo-cd chart: https://artifacthub.io/packages/helm/argo/argo-cd
configs:
  rbac:
    policy.csv: |
      p, CiByZXBvOm15LW9yZy9teS1yZXBvOnB1bGxfcmVxdWVzdBIOZ2l0aHViLWFjdGlvbnM, projects, get, my-project, allow
      p, CiByZXBvOm15LW9yZy9teS1yZXBvOnB1bGxfcmVxdWVzdBIOZ2l0aHViLWFjdGlvbnM, applications, get, my-project/*, allow

That long string can be found in logs of argocd-server. It can also be generated by using something like https://www.protobufpal.com/

Or, if anyone else also comes here looking for help for ArgoCD, you could also try upgrade to ArgoCD v3 instead where your config just needs to be this: (thanks to v3 actually using federated_claims.user_id)

# Helm values for the argo-cd chart: https://artifacthub.io/packages/helm/argo/argo-cd
configs:
  rbac:
    policy.csv: |
      p, repo:my-org/my-repo:pull_request, projects, get, my-project, allow
      p, repo:my-org/my-repo:pull_request, applications, get, my-project/*, allow

applejag avatar May 13 '25 17:05 applejag

Is the documentation going to be updated? It would be beneficial for anyone to find this information in the documentation, rather than in GitHub issues.

polasekr avatar Aug 06 '25 20:08 polasekr

Is the documentation going to be updated? It would be beneficial for anyone to find this information in the documentation, rather than in GitHub issues.

I created a PR to ArgoCD's documentation for this: https://github.com/argoproj/argo-cd/pull/22953 (still not merged)

But I don't believe there's any PRs for Dex's documentation

applejag avatar Aug 06 '25 21:08 applejag

@applejag, thanks for the research and sharing your findings, this is super helpful!

I am also setting up machine authentication for github-actions, and was looking into the origin of weird sub format returned by dex.

By the way, have you found if there's a way to "hide" the github-actions connector from being shown on the dex login page?

Context: I have argocd with built-in dex configured to have SSO using an OIDC connector. Click on "SSO LOGIN" button redirects me directly to the IDB login page. But after adding the second connector for github actions, now I get the "dex" login page with 2 buttons: login with oidc and login with github-actions

DEX actually allows redirecting to IDP login page if /api/dex/auth?connector_id=<con_id> is provided, but I could not find how to make argocd adds this QueryString parameter

antonu17 avatar Aug 14 '25 07:08 antonu17

By the way, have you found if there's a way to "hide" the github-actions connector from being shown on the dex login page?

I haven't found a way to do this either. We've just settled for teaching everyone at our company to click the "Okta" button instead of the "GitHub Actions" button.

Theoretically you could maybe add some proxy to Dex that overrides its /.well-known/openid-configuration path and maybe change the authorization_endpoint to return /auth/{con_id} instead of /auth, but that may mess up regular "GitHub Actions" auth then. As it stands now, there's no easy way to get around this.

I've been trying to get our company's sysadmins to see if we can set up OAuth 2.0 On-Behalf-Of Token Exchange in Okta (docs: https://developer.okta.com/docs/guides/set-up-token-exchange/main/), as that would remove the need for Dex and would mean we could use GitHub Actions IdP for other things too like Harbor or Kubernetes access. But while Okta has a free plan to test this out, adding this however to our main Okta org costs some money and our sysadmins have not been able to get a price tag from Okta's support yet so we're currently just waiting.

If the upstream IdP that you're using also supports OAuth 2.0 On-Behalf-Of Token Exchange then you could maybe go down that route.

However as it seems now, at our company we'll probably settle for something like this (as the Okta On-Behalf-Of thing doesn't seem to become a reality):

  • Keep using Dex with the tedious "multiple connectors" sign in page. It's not that annoying to work around.
  • For Harbor we'll probably just settle for using robot accounts with static credentials that we provide to our dev teams so they can store that in GitHub Actions secrets
  • For Kubernetes, we already use OIDC for human kubectl access, which works great. And in Kubernetes v1.30 the "Structured Authentication Configuration" graduated to beta (meaning its enabled by default) which allows for specifying multiple OIDC IdP (https://kubernetes.io/blog/2024/04/25/structured-authentication-moves-to-beta/), so we'll probably try out this to add GitHub Actions IdP in there.

applejag avatar Aug 14 '25 08:08 applejag

@phamviet funny, I'm debugging this atm too. Scary seeing "now" instead of a timestamp

Setting subject_token_type=urn:ietf:params:oauth:token-type:id_token as you suggested did fix it for me too! (and leave requested_token_type=urn:ietf:params:oauth:token-type:access_token)

One thing Dex does is it encodes the sub as base64-encoded protobuf message IDTokenSubject, as is explained in this ticket: #1719. Dex also sets federated_claims.user_id to the original sub, which servers should make use of.

Personally I've been trying to get GitHub Actions auth working for ArgoCD. ArgoCD does not make use of federated_claims.user_id, but instead it uses the sub. (This changed in ArgoCD v3: argoproj/argo-cd#20683, however I'm still stuck at ArgoCD v2 for the time being)

So I was also stuck for a little while getting ArgoCD auth working, but eventually I figured out that I had to do this:

Helm values for the argo-cd chart: https://artifacthub.io/packages/helm/argo/argo-cd

configs: rbac: policy.csv: | p, CiByZXBvOm15LW9yZy9teS1yZXBvOnB1bGxfcmVxdWVzdBIOZ2l0aHViLWFjdGlvbnM, projects, get, my-project, allow p, CiByZXBvOm15LW9yZy9teS1yZXBvOnB1bGxfcmVxdWVzdBIOZ2l0aHViLWFjdGlvbnM, applications, get, my-project/*, allow That long string can be found in logs of argocd-server. It can also be generated by using something like https://www.protobufpal.com/

Or, if anyone else also comes here looking for help for ArgoCD, you could also try upgrade to ArgoCD v3 instead where your config just needs to be this: (thanks to v3 actually using federated_claims.user_id)

Helm values for the argo-cd chart: https://artifacthub.io/packages/helm/argo/argo-cd

configs: rbac: policy.csv: | p, repo:my-org/my-repo:pull_request, projects, get, my-project, allow p, repo:my-org/my-repo:pull_request, applications, get, my-project/*, allow

There is perhaps another way that doesn't mess argocd-v2 rbac config with encoded user id like p, CiByZXBvOm15LW9yZy9teS1yZXBvOnB1bGxfcmVxdWVzdBIOZ2l0aHViLWFjdGlvbnM.

The trick is to configure dex to add groups claims using claimModifications.newGroupFromClaims. Example:

          - type: oidc
            id: github-actions
            name: github-actions
            config:
              issuer: https://token.actions.githubusercontent.com
              userNameKey: sub
              insecureSkipEmailVerified: true
              claimMapping:
                email: sub
              claimModifications:
                newGroupFromClaims:
                  - prefix: github-actions-runner-environment
                    delimiter: "::"
                    clearDelimiter: false
                    claims:
                      - runner_environment
                  - prefix: github-actions-repository
                    delimiter: "::"
                    clearDelimiter: false
                    claims:
                      - repository
                  - prefix: github-actions-job-workflow-ref
                    delimiter: "::"
                    clearDelimiter: false
                    claims:
                      - job_workflow_ref
                  - prefix: github-actions-sub
                    delimiter: "::"
                    clearDelimiter: false
                    claims:
                      - sub

With this config argocd account get-user-info looks like:

Logged In: true
Username: repo:org/repo:pull_request
Issuer: argocd-server/api/dex
Groups: github-actions-runner-environment::github-hosted,github-actions-repository::org/repo,github-actions-job-workflow-ref::org/repo/.github/workflows/ci.yml@refs/pull/1/merge,github-actions-sub::repo:org/repo:pull_request

Now you can refer groups in rbac config, example:

    rbac:
      policy.csv: |
        g, github-actions-runner-environment::github-hosted, role:readonly
        g, github-actions-sub::repo:org/repo:push, role:deploy

There's quite a lot claims available in the github actions jwt token to play with.

antonu17 avatar Aug 16 '25 08:08 antonu17

By the way, have you found if there's a way to "hide" the github-actions connector from being shown on the dex login page?

I haven't found a way to do this either. We've just settled for teaching everyone at our company to click the "Okta" button instead of the "GitHub Actions" button.

For visibility: https://github.com/argoproj/argo-cd/issues/24176

antonu17 avatar Aug 16 '25 14:08 antonu17