Queue Trigger - Consumption Plan - Goes to sleep and does not wake up

[MLRichmond]

Check for a solution in the Azure portal

We have a simple Azure Function that is running on a Consumption ASP. The function runs fine when initially deployed or when "poked" from the Azure Portal - but after a period of time, goes to sleep and does not wake up, even if new messages arrive in the corresponding queue.

Please provide the following:

• Timestamp: 10/4/2024 2:50 PM CST
• Function App version:
• Function App name: KermitFunctions
• Function name(s) (as appropriate): QueuePackager
• Invocation ID:
• Region: Central US

Repro steps

Provide the steps required to reproduce the problem:

1. QueueTrigger function using Managed Identity and App Settings Connection information.
2. Deploy function
3. Add msg to queue
4. Msg processes.
5. Wait 20-30 minutes (AppInsights log will show several "Disposing ScriptHost" messages)
6. Add msg to queue
7. Msg is never processed.

Expected behavior

Expected that QueueTrigger would eventually wake up and process the incoming message.

Actual behavior

QueueTrigger never wakes up.

Known workarounds

Go to Azure Portal and view Function. It will wake up and process.

Related information

[bhagyshricompany]

Hi @MLRichmond thanks for reporting can you try with same connection string ?

[MLRichmond]

@bhagyshricompany Sorry, what do you mean by same connection string? Re-testing this morning, it's still not waking up.

[bhagyshricompany]

you are using identity for connection option? please share log details.Thanks

[MLRichmond]

Basically after 20 minutes or so of not receiving a queue message, we see this: Then after that, if send in a new message, it never wakes up.

[Angelm116]

I am seeing a similar issue, but with a timer trigger function. The timer is set to trigger every 5 minutes. The function goes to sleep 20 minutes after deployment, and the timer trigger no longer fires. Similar to your case, the trigger starts firing again when I open the function in Azure Portal. When I go to Developer tools -> Kudu -> LogFiles, I see this:

[andrewkittredge]

we have the same problem

[bhagyshricompany]

I checked and seems working fine.

[MLRichmond]

@bhagyshricompany yes, on an initial run it works fine. The point is that when hosted in azure, after 20 minutes or so of inactivity, it gets out to sleep and never wakes again, even if additional messages come in.

Running it in a debugger isn't comparable to the failure mode. That, of course works fine.

[bhagyshricompany]

yes I got it.Will update you on this shortly.

[baumatron]

I'm seeing the same issue for the following configuration:

1. C# .NET 6 in-proc queue trigger, Microsoft.NET.Sdk.Functions 4.6.0 SDK, Microsoft.Azure.WebJobs.Extensions.Storage 5.3.3
2. Premium EP3 hosting plan.
3. Messages are processed successfully for hours or even days after initial startup
4. Eventually the host listener is stopped and the function is never triggered again despite there being queue messages present. Invocations for the queue trigger. Last processed was at 2024-11-09 7:44:23 PM UTC-8

Current messages in queue. Earliest was inserted on 2024-11-09 7:47 PM UTC-8

Logs from the cloud role name show that after processing the last trigger the QueueListener was stopped, and finally it acquired the host lock blob lease despite not listening for queue messages. After the host lock was acquired, no more messages are triggered. There are no errors and the portal shows the app as running.

As an additional data point, I have other function apps with queue triggers as part of this deployment that are running on dedicated hosting plans and are not experiencing this problem.

[bhagyshricompany]

we are checking once it s done will update soon.Thanks

[tabiday]

I have the same issue, with an app on consumption plan triggered by blob storage. Please update as soon as possible

[Peter-Juhasz]

We are experiencing the same symptoms:

• Both on Flex Consumption Plan (Linux) and App Service Plan (Linux)
• Always-Ready enabled for FCP and Always On for ASP
• Both in France Central and North Europe
• Both with Storage Queue and Timer triggers
• We use Managed Identity to the Storage Queue
• .NET 9 Isolated model
• Usually after a deployment, sometimes on its own, it stops working
• Portal shows everything is fine, Functions green "Enabled", no logs indicating any error

The triggers stop working, messages are piling up in the queues, no errors or any logs, only manually restarting the app solves the issue, which is unacceptable for production workloads, and it impacts our customers.

@bhagyshricompany what is the ETA for the resolution? Or is there any workaround exists?

[surgupta-msft]

Have you all started seeing this behavior recently or that was always the case?

[MLRichmond]

@surgupta-msft As far as I have been able to observe, this has always been the case.

[Dolphinsimon]

I have met the same issue; the timer trigger is set to trigger every 5 minutes. The function goes to sleep after the trigger invoked 5 times. Creating new functions in the same resource group won't solve the issue. We have found that we must create new functions in other resource groups to fix the issue.

[Dolphinsimon]

We have found that all the problematic functions are hosted in the same HomeStamp. If we are creating new functions in other resource groups (Also other HomeStamp) will fix the issue. The issue appears to be more a deployment problem rather than a code issue in function host.

[satvu]

I'm not sure all of the different comments on this thread are related to the original issue, but I think the original issue could be related to https://github.com/Azure/azure-functions-host/issues/8261 - @MLRichmond can you try the mitigation described in that thread and see if it works? The steps include:

• Explicitly defining the Connection property
• Reviewing and adding roles as needed (Data Owner, Data Sender, Data Receiver)

[dreinhardt89]

This happens periodically for us as well.

• Function stops consuming messages in queue.
• User reports issue.
• We go to visit the function in the azure portal which wakes it back up and gets it consuming messages again.