Enable using the same service connection approach as Power Platform Build Tools

[devkeydet]

Please consider enhancing the tasks to support using the same connection approach using by the build tools: https://docs.microsoft.com/en-us/power-platform/alm/devops-build-tools#connection-to-environments Many people will use both sets of tasks in pipelines. I know I do :). Using the same approach would enable tasks using a single connection vs having to maintain two connection approaches in a pipeline.

[lar-mar]

Then I would rather like to see a changed approach by Microsoft. The connection string approach by @WaelHamze makes it more easy for providing a generic pipeline which can be reused with really less efforts in other DevOps projects than the approach by Microsoft.

[devkeydet]

Thanks for the feedback. I have passed this on to the Power Platform Build Tools team.

[mattp65]

@devkeydet Is the build tools connection method officially 'supported' by all the underlying tasks? These tools wrapper existing APIs that per the doc need a connection string. Personally, I'm not a fan of the connection used by the power platform build tools. It is too locked in and not flexible as @lar-mar points out.

[devkeydet]

@mattp65 - Yes, the tools are GA now: https://powerapps.microsoft.com/en-us/blog/announcing-general-availability-of-microsoft-power-platform-build-tools/, including service principal connection. The new service principal connection docs are here: https://docs.microsoft.com/en-us/power-platform/alm/devops-build-tools#configure-service-connections-using-a-service-principal.

[WaelHamze]

@devkeydet

I think having a consistent way of connecting would make it easier for consumers.

However there is a draw back...

The Power Platform connection is a special type. Need to check if this can be referenced from another tool, this would also put a hard dependency on another tool which could result in impact to consumer if PPBT makes breaking changes.

I find the connection string useful as you can easily supply/change it via library or Key Vault without impacting any of your pipelines. You can also re-use certain secret across environments without having to go through all your connections if you have many environments.

Let me know if you still think we should change this.

[devkeydet]

I think the how for allowing consumers less important. What's most important is that consumers don't have to maintain two sets of connections. Appreciate all the feedback here. I share it with the Power Platform Build Tools team and brainstorm on thoughts on the best way to collaborate on the ultimate goal: pipeline author doesn't need to maintain connection information twice when using both tasks.

[mattp65]

@devkeydet I would add that it is also important to be able to leverage variables for the credentials so something like Keyvault can be used. For a release pipeline that targets multiple environments, the environment specific variables make it straightforward to clone the tasks between Envs without really changing any of the task definitions when using a connection string.

[WaelHamze]

@devkeydet @mattp65 thanks for the useful feedback...

@devkeydet please let us know once you have discussion with the Power Platform Build Tools team.

[devkeydet]

Sorry it has taken so long to follow up. I've had a few discussions with the built tools team and also needed to wrap my head around AzDO service connections more deeply (and find the time to do so).

AzDO service connections, by design, provide a separation of concerns between project administrators and pipeline authors. They allow project administrators to create service connections, while only allowing pipeline authors to point to those service connections, but otherwise have no visibility into the service connection. Through this separation of concerns, there is also a layer of security. See: https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#secure-a-service-connection

That's why the official build tools do not use connection strings and instead use the AzDO prescribed approach to connecting to external services: service connections.

To the point about leveraging variables for credentials using KeyVault. Even the approach to getting values out of KeyVault uses Service Connections.

So, in the end, to get something out of KeyVault, you have to store some secret in the service connection that allows you to access the KeyVault service. So in this particular case, I don't see the benefit of using KeyVault values to compose a connection string using the secret over just putting the secret in the service connection. However, I'm looking for feedback to better understand others point of view.

In the end, service connections are the prescribed way, in Azure DevOps, to connect to external services. Therefore, the build tools team feedback is that supporting connection strings in the Power Platform Build Tools is not something they are considering.

[WaelHamze]

@devkeydet / all

Thanks for looking into this.

Regarding separation of concerns, if I am not mistaken, service connections use libraries under the hood. So whether you storing secrets in service connection or libraries. You are achieving the same thing. Admins can still manage secrets in the libraries. The connection string is just a serialised version of a service connection.

I guess the advantage of storing secrets in keyVault is that you can have these stored central and benefit from the additional features that KeyVault offers. This also prevents multiple copies of the same secret in multiple places.

Secret rotation is one of the controls used to reduce risk. If you have 20 environments and 20 service connections using same secret, now you will have to update all of the service connections instead of one value in library or Key Vault.

I am not saying that connection strings are the solutions to everything and suggest that PP Build Tools adopt this. Just trying to raise a few points that might be worth considering during implementation. I am all up for abstracting connections. Many add-ons like SonarQube use this mechanism too.

But even if we were to switch Power DevOps Tools to use service connections. It would need to have its own service connection to avoid hard dependency on another tool that I don't have control over changes/releases. So developers will have to create two service connections for the same thing.

I see the new GitHub actions have parameters for each of the different parts of the credentials. With secret values managed in the secrets area.

Let me know your thoughts.

[devkeydet]

@WaelHamze - All valid points. However, for separation of concerns in this case, it sounds more like an ask of the AzDO team that owns Service Connections to support pulling values from KeyVault or variables through libraries (which can already use KeyVault). That way, you could have Service Connections in different projects (or even the same project) that that pull the same values from KeyVault. That way, you wouldn't need to take a dependency on the Service Connection type shipped with the Power Platform Built Tools. You could have your own type, but connect to a service in a fashion that is consistent with how Microsoft authored tasks, including Azure, etc., use connections. And in the end, the pipeline author can achieve the ultimate goal of not having to maintain multiple values to connect to Power Platform Environments across two task libraries. Thoughts?

[kashanico]

I'd like to add my thoughts on this as I've been creating pipelines using both sets of tools. It would be much more uniform if all the creds were managed by the Service Connections feature of Azure Devops. Most extensions are going that route and it is a better experience, especially for templating pipelines if we are not hard coding connection strings in the yaml.

[scubaracer]

I personally don't like the service connections, i think the "string" values offer more flexibility. But what i would like is the support of the service principal for creating/copying instances. As most of the customers where we use the tooling don't want to use username/password. And then we have to use the Microsoft Power Platform tools :-(.

[jshield-AFF]

A solution is to provide another market extension that provides a task that consumes the PPBT service connection as a dependency, extracts the secrets and stores them in a secret variable in the pipeline conforming to the Connection String format that this set of tasks expect, which can then be used to bridge the worlds without impacting existing consumers of either tasks.

This is the solution we are implementing at my place of work, i'll see if can get approval to open source it and publish it to the public on the marketplace.

I used this approach to allow me to run arbitrary powershell inside of the PPBT Service Connection context, similar to the Azure PowerShell tasks.

[devkeydet]

It's been a long time since I raised this issue. Closing it because an upcoming release of the PPBT will have a task that gives you the information you need to use the Service Connection as the source of truth and coexist with tools used in script tasks including PowerShell cmdlets, other cli, and this task library which expect credential values or connection string format. See: https://github.com/microsoft/powerplatform-build-tools/tree/main/src/tasks/set-connection-variables/set-connection-variables-v2