Azure.Identity 1.6.0 - AuthenticationFailedException: ManagedIdentityCredential authentication failed: No Managed Identity found

[phil000]

trafficstars

Library name and version

Azure.Identity 1.6.0

Describe the bug

We make use of Azure Key Vault and use a system-assigned managed identity of the Azure Function to connect to KV at runtime.

With Azure.Identity 1.6.0 we get "AuthenticationFailedException: ManagedIdentityCredential authentication failed: No Managed Identity found for specified ClientId/ResourceId/PrincipalId"

Reverting back to Azure.Identity 1.5.0 fixes it with no other changes our side.

We are using 'DefaultAzureCredential' to provide credentials to Key Vault SecretClient.

DefaultAzureCredentialOptions credentialOptions = new DefaultAzureCredentialOptions()
{
	ExcludeAzureCliCredential = true,
	ManagedIdentityClientId = _configuration.ManagedIdentityClientId
};
SecretClientOptions clientOptions = GetClientOptions();
_client = new SecretClient(new Uri(_configuration.KeyVaultUrl), new DefaultAzureCredential(credentialOptions), clientOptions);

See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
Exception: Azure.Identity.AuthenticationFailedException: ManagedIdentityCredential authentication failed: No Managed Identity found for specified ClientId/ResourceId/PrincipalId.
Status: 400 (Bad Request)

   at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken)
   at Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
   at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
   at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage)
   at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
   at Azure.Identity.ManagedIdentityCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
   at Azure.Identity.DefaultAzureCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(TokenRequestContext context, Boolean async, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AuthenticateAndAuthorizeRequest(HttpMessage message, TokenRequestContext context)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestInternal(HttpMessage message, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequest(HttpMessage message)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Core.Pipeline.RedirectPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.HttpPipeline.SendRequest(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest[TResult](RequestMethod method, Func`1 resultFactory, CancellationToken cancellationToken, String[] path)
   at Azure.Security.KeyVault.Secrets.SecretClient.GetSecret(String name, String version, CancellationToken cancellationToken)
   at Onsend.Lib.Core.KeyVault.KeyVaultManager.GetSecret(String secretName) in /_/Onsend.Lib/Core/KeyVault/KeyVaultManager.cs:line 117
   at Onsend.Lib.Core.KeyVault.KeyVaultSecretProvider.GetSecret(String key, Boolean required) in /_/Onsend.Lib/Core/KeyVault/KeyVaultSecretProvider.cs:line 59

Expected behavior

Gets managed identity.

Actual behavior

Throws exception. AuthenticationFailedException: ManagedIdentityCredential authentication failed: No Managed Identity found for specified ClientId/ResourceId/PrincipalId

Reproduction Steps

Upgrade from Azure.Identity 1.5.0 to 1.6.0

Environment

Azure Functions: FUNCTIONS_EXTENSION_VERSION: ~4 FUNCTIONS_WORKER_RUNTIME: dotnet-isolated

[phil000]

Here are the variables set in the Azure environment: MSI_ENDPOINT = http://127.0.0.1:41538/MSI/token/ IMDS_ENDPOINT (not present) AZURE_POD_IDENTITY_AUTHORITY_HOST (not present)

[jsquire]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[juan-manuel-vergara]

Hi, it would be great to have this solved, I have this problem too. thanks

[phil000]

I've enabled AzureEventSourceListener diagnostics logging while using Azure.Identity 1.6.1, and this is what I got.

Reverting back to Azure.Identity 1.5.0 and this all works fine, with no exception occurring.

3 fields were logged

1. OSThreadId
2. EventSource.Name
3. message

8112 Azure-Identity DefaultAzureCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: fcea3657-3d86-4b65-9d46-62d6b36adce4

8112 Azure-Identity EnvironmentCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: fcea3657-3d86-4b65-9d46-62d6b36adce4

8112 Azure-Identity EnvironmentCredential.GetToken was unable to retrieve an access token. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: fcea3657-3d86-4b65-9d46-62d6b36adce4 Exception: Azure.Identity.CredentialUnavailableException (0x80131500): EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot

8112 Azure-Identity ManagedIdentityCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: fcea3657-3d86-4b65-9d46-62d6b36adce4

8112 Azure-Identity ManagedIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: fcea3657-3d86-4b65-9d46-62d6b36adce4 Exception: Azure.Identity.AuthenticationFailedException (0x80131500): ManagedIdentityCredential authentication failed: No Managed Identity found for specified ClientId/ResourceId/PrincipalId. Status: 400 (Bad Request)

[phil000]

From the above logging it seemed the 'EnvironmentCredential' was having an issue -> CredentialUnavailableException - "Environment variables are not fully configured".

After excluding environmental credentials we still are unable to retrieve a token and the issue seems to be squarely with ManagedIdentityCredential.GetToken -> Azure.Identity.AuthenticationFailedException (0x80131500)

This is in an Azure Function environment running on a Windows-based app service plan, using a timer triggered function, and: FUNCTIONS_EXTENSION_VERSION: ~4 FUNCTIONS_WORKER_RUNTIME: dotnet-isolated

[phil000]

Some environmental variables that are set in the functions environment:

IDENTITY_ENDPOINT = http://127.0.0.1:41538/MSI/token/ IDENTITY_HEADER = 78xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

MSI_ENDPOINT = http://127.0.0.1:41538/MSI/token/ MSI_SECRET = 78xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

'IDENTITY_HEADER' and 'MSI_SECRET' have the same value.

Poking about in the code...

In Azure.Identity 1.5.0 AppServiceV2019ManagedIdentitySource is in the assembly but doesn't seem to be returned from ManagedIdentityClient.SelectManagedIdentitySource

However, when upgrading to Azure.Identity 1.6.0 it is returned I guess (as the environmental variables for it's use are set)

After that a request is created and sent, but a 400 (BadRequest) response is returned.

[christothes]

Hi @phil000 In the original issue description, you mention wanting to use the system-assigned managed identity, but you are also specifying a ManagedIdentityClientId. To get the system-assigned identity, you should remove the clientId setting.

[phil000]

Hello @christothes We have been using system managed identities for about a year now.

Having a look I think this is what may have happened:

We started Azure Functions using user-managed identities but swapped to system-managed identities about a year ago as we wanted to use Key Vault References for secrets that are basically expected to be in the Functions configuration.

Everything seemed to work fine but we did not remove the ManagedIdentityClientId that is passed to our KeyVault SecretClient that is used to look up other secrets at runtime. In most cases the 'ManagedIdentityClientId' points to an identity that doesn't exist, but this all worked fine anyway.

Upgrading from Azure.Identity 1.5.0 to 1.60 seemed to expose this configuration issue, but it wasn't obvious it was a configuration issue as the only change was a package version update that broke us.

Removing the ManagedIdentityClientId seems to have worked and we have upgraded to 1.7.0 now without the former issue occurring.

[ghost]

Hi @phil000. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

[ghost]

Hi @phil000, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.