🐛 Bug Report: Azure Devops Repo Picker doesn't work with user credentials

[afscrome]

📜 Description

The repo picker doesn't work out of the box with Azure devops due to the scopes in the default ScmAuthApi implementation not fully qualifying the scope names. As the scopes aren't fully qualified, backstage tries to request the scopes from the Microsoft Graph API rather than Azure Devops, which then fails as the requested scopes don't exist in Microsoft Graph.

You can see the kubernetes auth provider as an example of one that fully qualifies its scopes - https://github.com/backstage/backstage/blob/f600aa87a3b3f7700544653b8560c86cba3cd479/plugins/kubernetes-react/src/kubernetes-auth-provider/AksKubernetesAuthProvider.ts#L35-L37

I see two possible solutions to this problem

1. Fully qualify the azure devops scopes in the default ScmAuth implementation. (i.e. vso.code --> 499b84ac-1321-427f-aa17-267ca6975798/vso.code) https://github.com/backstage/backstage/blob/f600aa87a3b3f7700544653b8560c86cba3cd479/packages/integration-react/src/api/ScmAuth.ts#L202-L208
2. For the Microsoft Auth API, change the getAccessToken method to accept a resource as well as scopes argument. When this resource is provided, it can be used for resourceForScopes, as well as be prepended to all requested scopes.

👍 Expected behavior

Should get an Azure Devops scoped token from the Microsoft authentication apis.

👎 Actual Behavior with Screenshots

Authentication failed, AADSTS650053: The application REDACTED' asked for scope 'vso.build' that doesn't exist on the resource '00000003-0000-0000-c000-000000000000'. Contact the app vendor. Trace ID: 7a2bdb5f-c0cc-4856-af8a-e3594b4d6900 Correlation ID: 65074d10-81f8-4da5-8843-5c1338e3a900 Timestamp: 2024-02-01 15:35:08Z

(00000003-0000-0000-c000-000000000000 is the clientId for Microsoft Graph.

👟 Reproduction steps

1. Configure the microsoft auth provider ( https://backstage.io/docs/auth/microsoft/provider/ )
2. Configure a software template with a repo picker (see example below)
3. Fill in the required details for the repo picker, and follow the auth prompt

parameters:
  - title: Fill in some steps
    properties:
      repoUrl:
        title: Repository Location
        type: string
        ui:field: RepoUrlPicker
        ui:options:
          allowedHosts:
            - dev.azure.com        
          requestUserCredentials:
            secretsKey: USER_OAUTH_TOKEN
            additionalScopes:
              azure: []
steps:
  - id: fetch-base
    name: Fetch Base
    action: fetch:template
    input:
      url: ./template
      values:
        name: ${{parameters.name}}

📃 Provide the context for the Bug.

No response

🖥️ Your Environment

OS: Linux 5.15.133.1-microsoft-standard-WSL2 - linux/x64 node: v20.10.0 yarn: 1.22.21 cli: 0.25.0 (installed) backstage: 1.21.1

Dependencies: @backstage/app-defaults 1.4.6 @backstage/backend-app-api 0.5.10, 0.5.9 @backstage/backend-common 0.20.0, 0.20.1 @backstage/backend-dev-utils 0.1.2, 0.1.3 @backstage/backend-openapi-utils 0.1.1 @backstage/backend-plugin-api 0.6.8, 0.6.9 @backstage/backend-tasks 0.5.13, 0.5.14 @backstage/catalog-client 1.5.1, 1.5.2 @backstage/catalog-model 1.4.3 @backstage/cli-common 0.1.13 @backstage/cli-node 0.2.1, 0.2.2 @backstage/cli 0.25.0 @backstage/config-loader 1.6.0, 1.6.1 @backstage/config 1.1.1 @backstage/core-app-api 1.11.2 @backstage/core-compat-api 0.1.0 @backstage/core-components 0.13.9 @backstage/core-plugin-api 1.8.1 @backstage/e2e-test-utils 0.1.0 @backstage/errors 1.2.3 @backstage/eslint-plugin 0.1.4 @backstage/frontend-plugin-api 0.4.0 @backstage/integration-aws-node 0.1.8 @backstage/integration-react 1.1.22 @backstage/integration 1.8.0 @backstage/plugin-api-docs 0.10.2 @backstage/plugin-app-backend 0.3.56 @backstage/plugin-app-node 0.1.8 @backstage/plugin-auth-backend-module-atlassian-provider 0.1.0 @backstage/plugin-auth-backend-module-gcp-iap-provider 0.2.2 @backstage/plugin-auth-backend-module-github-provider 0.1.5 @backstage/plugin-auth-backend-module-gitlab-provider 0.1.5 @backstage/plugin-auth-backend-module-google-provider 0.1.5 @backstage/plugin-auth-backend-module-oauth2-provider 0.1.5 @backstage/plugin-auth-backend-module-oauth2-proxy-provider 0.1.0 @backstage/plugin-auth-backend-module-okta-provider 0.0.1 @backstage/plugin-auth-backend 0.20.2 @backstage/plugin-auth-node 0.4.2, 0.4.3 @backstage/plugin-azure-devops-backend 0.5.0 @backstage/plugin-azure-devops-common 0.3.2 @backstage/plugin-azure-devops 0.3.10 @backstage/plugin-catalog-backend-module-azure 0.1.27 @backstage/plugin-catalog-backend-module-msgraph 0.5.15 @backstage/plugin-catalog-backend-module-scaffolder-entity-model 0.1.5 @backstage/plugin-catalog-backend 1.16.0 @backstage/plugin-catalog-common 1.0.19 @backstage/plugin-catalog-graph 0.3.2 @backstage/plugin-catalog-node 1.6.0 @backstage/plugin-catalog-react 1.9.2 @backstage/plugin-catalog 1.16.0 @backstage/plugin-entity-validation 0.1.13 @backstage/plugin-events-node 0.2.17 @backstage/plugin-explore-backend 0.0.18 @backstage/plugin-explore-common 0.0.2 @backstage/plugin-explore-react 0.0.34 @backstage/plugin-explore 0.4.14 @backstage/plugin-github-actions 0.6.9 @backstage/plugin-home-react 0.1.6 @backstage/plugin-home 0.6.0 @backstage/plugin-org 0.6.18 @backstage/plugin-permission-common 0.7.11, 0.7.12 @backstage/plugin-permission-node 0.7.19, 0.7.20 @backstage/plugin-permission-react 0.4.18 @backstage/plugin-proxy-backend 0.4.6 @backstage/plugin-scaffolder-backend-module-azure 0.1.0 @backstage/plugin-scaffolder-backend-module-bitbucket 0.1.0 @backstage/plugin-scaffolder-backend-module-gerrit 0.1.0 @backstage/plugin-scaffolder-backend-module-github 0.1.0 @backstage/plugin-scaffolder-backend-module-gitlab 0.2.11 @backstage/plugin-scaffolder-backend 1.19.2 @backstage/plugin-scaffolder-common 1.4.4 @backstage/plugin-scaffolder-node 0.2.9 @backstage/plugin-scaffolder-react 1.7.0 @backstage/plugin-scaffolder 1.17.0 @backstage/plugin-search-backend-module-catalog 0.1.12 @backstage/plugin-search-backend-module-explore 0.1.13 @backstage/plugin-search-backend-module-pg 0.5.17 @backstage/plugin-search-backend-module-techdocs 0.1.12 @backstage/plugin-search-backend-node 1.2.12, 1.2.13 @backstage/plugin-search-backend 1.4.8 @backstage/plugin-search-common 1.2.10, 1.2.9 @backstage/plugin-search-react 1.7.4 @backstage/plugin-search 1.4.4 @backstage/plugin-tech-radar 0.6.11 @backstage/plugin-techdocs-backend 1.9.1 @backstage/plugin-techdocs-module-addons-contrib 1.1.3 @backstage/plugin-techdocs-node 1.11.0 @backstage/plugin-techdocs-react 1.1.14 @backstage/plugin-techdocs 1.9.2 @backstage/plugin-user-settings 0.7.14 @backstage/release-manifests 0.0.11 @backstage/test-utils 1.4.6 @backstage/theme 0.5.0 @backstage/types 1.1.1 @backstage/version-bridge 1.0.7

👀 Have you spent some time to check if this bug has been raised before?

• [X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• [X] I have read the Code of Conduct

Are you willing to submit PR?

Yes I am willing to submit a PR!

[Rugvip]

@jamieklassen any chance there's something you spot here at a glance? 😅 🙏

[Phiph]

I've just hit this issue :) Thanks for raising it!

[camilaibs]

Thanks everyone, we're looking for contributions. I was wondering if you had any suggestions, @awanlin?

[Phiph]

I'm going to give it a go today, as it might also solve another issue I have!

It feels hacky but I think a simple of statement should do the trick

[awanlin]

@Phiph, not sure what you mean? Would not just running with option 1 be the easy solution - adding the GUID to each scope?

[Phiph]

Would those GUID scopes work with Azure DevOps server (self hosted) too?

[afscrome]

OAuth authentication isn't available on Azure Devops Server, so this integration is specific to Azure Devops Services - https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/oauth?view=azure-devops .

[awanlin]

Awesome, thanks @afscrome, that's what I was going to say but will much less certainty!

[Phiph]

Great thanks team I've got enough information to make sure I get it right.

I think once this work has been complete I may be able to continue where I left off with this PR: https://github.com/backstage/backstage/pull/20923

[Phiph]

Okay, I've had a go - and got some code here https://github.com/Phiph/backstage/tree/bug/ado-scopes

While testing the change I get the following:

I don't suppose you've got any ideas on what it could be.

To test I have created an App registration with the above api's available to it.

I then run the following:

 export AUTH_MICROSOFT_CLIENT_ID=<My-Id>
 export AUTH_MICROSOFT_CLIENT_SECRET=<My-Secret>
 export AUTH_MICROSOFT_TENANT_ID=<My-Tenant>

I then include a template that uses requestUserCredentials then yarn dev

[afscrome]

I’ve been encountering this same issue - it broke around the time I upgraded to the new backend. I’ve worked around this for now by reverting to the old with backend - using the old auth.ts file from the old back end, and registering that with useLegacyPlugin. Haven’t found time to debug the root cause in the new backend.

[Phiph]

Oh cool cheers I'll try that now - thanks Alex.

Is the new backend system ready for general use yet? If not any objections to raising the PR?

[Phiph]

Okay looks like I'm using the old backend to develop with anyway yarn dev I'm going to spend a bit of time tonight and tomorrow to figure out what it is.

[benjdlambert]

Added the auth label to this too as it looks like we've got some form of bug to do with the microsoftAuthProvider too right in the new backend system?

Just wanna clarify what's happening:

• legacyPlugin() with the new backend system works
• using the microsoftAuthProvider module directly using the new backend system throws `cannot read property of undefined 'emails'?

Thinking that maybe it could be best to raise this as a separate issue actually?

[afscrome]

I've logged the auth issue as #23032. Can discuss that issue further on that issue.

@benjdlambert - can you remove the auth label from this issue, and add it to #23032

[Phiph]

thanks @afscrome for fixing the core issue! I'll pick up this task later.