azure-functions-python-worker icon indicating copy to clipboard operation
azure-functions-python-worker copied to clipboard

BUG In Azure Function Trigger with Managed Identity

Open MiguelElGallo opened this issue 1 year ago • 2 comments

Expected Behavior

If you want to use managed identity for a function to have a trigger binding for event hub you can do:

Python

bp = func.Blueprint()
consumergroup = "$Default"
@bp.event_hub_message_trigger(
    arg_name="event",
    event_hub_name="name",   
    connection="ehconnection",   
    consumer_group=consumergroup)

local.settings.json (for running local)

localsettings.json(for running locally)
  "ehconnection__fullyQualifiedNamespace": "<namespace>.servicebus.windows.net",
 "ehconnection__credential": "managedIdentity",
 "ehconnection__clientId": "<clientId>"`

This does not work

Actual Behavior

It tries to connect to 169.254.169.254 which is a know address for Azure VMs. But it seems the Azure Function (local) runtime doest no provide that. (see log below)

Most probably in the Azure runtime it will work, but locally it does not work.

Note: In "ehconnection__clientId": "<clientId>" I have set my user id. I obtained it via : az ad signed-in-user show and field id from there.

[2024-09-03T10:17:58.894Z] Request [e98db5b7-e2aa-486e-be42-6180b172abbb] exception Azure.RequestFailedException: Operation timed out (169.254.169.254:80)
[2024-09-03T10:17:58.895Z]  ---> System.Net.Http.HttpRequestException: Operation timed out (169.254.169.254:80)
[2024-09-03T10:17:58.895Z]  ---> System.Net.Sockets.SocketException (60): Operation timed out
[2024-09-03T10:17:58.895Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
[2024-09-03T10:17:58.895Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
[2024-09-03T10:17:58.895Z]    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
[2024-09-03T10:17:58.895Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    --- End of inner exception stack trace ---
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
[2024-09-03T10:17:58.896Z]    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
[2024-09-03T10:17:58.897Z]    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-09-03T10:17:58.897Z]    --- End of inner exception stack trace ---
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-09-03T10:18:05.586Z] Request [e98db5b7-e2aa-486e-be42-6180b172abbb] attempt number 4 took 75.0s
[2024-09-03T10:18:05.586Z] Request [e98db5b7-e2aa-486e-be42-6180b172abbb] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED&client_id=REDACTED

Steps to Reproduce

1 Write an azure function that has Azure EventHub inbound binding 2 configure it to use managed identity 3 try to run it locally

Relevant code being tried

No response

Relevant log output

No response

requirements.txt file

No response

Where are you facing this problem?

Local - Core Tools

Additional Information

No response

MiguelElGallo avatar Sep 03 '24 10:09 MiguelElGallo

Update:

If I remove "ehconnection__credential": "managedIdentity", "ehconnection__clientId": "<clientId>"

Then I will get the following error when running locally, but then it will retry automatically , and it will work.

It seems some processes are faster than others.

[2024-09-03T11:38:31.361Z] False MSAL 4.60.3.0 MSAL.CoreCLR .NET 6.0.31 MacOS [2024-09-03 11:38:31Z - b794dc2f-386f-49ff-a11e-fa4388920033] Exception type: Azure.Identity.CredentialUnavailableException
[2024-09-03T11:38:31.361Z] ---> Inner Exception Details
[2024-09-03T11:38:31.361Z] Exception type: Azure.RequestFailedException
[2024-09-03T11:38:31.361Z] Date:Tue, 03 Sep 2024 11:38:31 GMT
[2024-09-03T11:38:31.361Z] ---> Inner Exception Details
[2024-09-03T11:38:31.361Z] Connection:keep-alive
[2024-09-03T11:38:31.362Z] Exception type: System.Net.Http.HttpRequestException
[2024-09-03T11:38:31.362Z] Keep-Alive:REDACTED
[2024-09-03T11:38:31.362Z] 
[2024-09-03T11:38:31.362Z] ---> Inner Exception Details
[2024-09-03T11:38:31.362Z] Exception type: System.Net.Sockets.SocketException
[2024-09-03T11:38:31.362Z] To see full exception details, enable PII Logging. See https://aka.ms/msal-net-logging
[2024-09-03T11:38:31.362Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
[2024-09-03T11:38:31.362Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ConnectAsync(Socket socket)
[2024-09-03T11:38:31.362Z]    at System.Net.Sockets.Socket.ConnectAsync(EndPoint remoteEP, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
[2024-09-03T11:38:31.363Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.<>c__DisplayClass75_0.<CheckForHttp11ConnectionInjection>b__0()
[2024-09-03T11:38:31.363Z]    at System.Threading.Tasks.Task`1.InnerInvoke()
[2024-09-03T11:38:31.363Z]    at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)

MiguelElGallo avatar Sep 03 '24 11:09 MiguelElGallo

Hi @MiguelElGallo thanks for update will check update you.

bhagyshricompany avatar Oct 17 '24 13:10 bhagyshricompany

Hello @MiguelElGallo could you please let me know if the issue still exists

JAdluri avatar Apr 08 '25 11:04 JAdluri

closing as there is no recent activity.

JAdluri avatar Apr 10 '25 08:04 JAdluri