TimerTrigger in Webjobs SDK does not support managed identity

[vlkchris]

We have a project

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <RootNamespace>Vlk.HrServices.Api.HrIntegration</RootNamespace>
    <UserSecretsId>2b57ac3c-5825-44df-ac93-078f754f2c38</UserSecretsId>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="AspNetCore.HealthChecks.UI.Client" Version="6.0.2" />
    <PackageReference Include="Azure.Extensions.AspNetCore.Configuration.Secrets" Version="1.2.1" />
    <PackageReference Include="Azure.Storage.Common" Version="12.12.0" />
    <PackageReference Include="Azure.Storage.Queues" Version="12.11.0" />
    <PackageReference Include="CsvHelper" Version="27.2.1" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions" Version="4.0.1" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage" Version="5.0.1" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage.Queues" Version="5.0.1" />
    <PackageReference Include="Microsoft.Extensions.Hosting" Version="6.0.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="6.0.0" />
    <PackageReference Include="Serilog.AspNetCore" Version="4.1.0" />
    <PackageReference Include="Serilog.Sinks.Console" Version="4.0.1" />
  </ItemGroup>

In this project we have configured webjobs as follows, in Program.cs

builder.Host.ConfigureWebJobs(b =>
{
    b.AddAzureStorageCoreServices();
    b.AddAzureStorageBlobs(options => { options.MaxDegreeOfParallelism = 1; });
    b.AddAzureStorageQueues(options =>
    {
        options.BatchSize = 1;
        options.MaxDequeueCount = 2;
    });
    b.AddTimers();
});

We have been using BlobTrigger and QueueTrigger succesfully, using the simplified storage account configuration in appsettings.json

"AzureWebJobsStorage": {
    "accountName": "hrintegrationtstsa"
  },

We are using managed identity for the webapp and have assigned proper roles to the webapp's identity and everything is working fine.

Now we have the need to add TimerTrigger functionality to this project. However it seems that the TimerTrigger is not compatible with the connection information in our appsettings.json. On startup it complains about a null connectionstring.

[14:33:19 DBG] The 'RunAsync' timer is using the schedule 'Cron: '0 0 0,4,8,12,16,20 * * 1-5'' and the local time zone: '(UTC+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna'
[14:33:19 ERR] The listener for function 'NsExportSubscriber.RunAsync' was unable to start.
Microsoft.Azure.WebJobs.Host.Listeners.FunctionListenerException: The listener for function 'NsExportSubscriber.RunAsync' was unable to start.
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'connectionString')
   at Microsoft.Azure.Storage.CloudStorageAccount.Parse(String connectionString)
   at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.get_TimerStatusDirectory() in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 73
   at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.GetStatusBlobReference(String timerName) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 144
   at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.GetStatusAsync(String timerName)
   at Microsoft.Azure.WebJobs.Extensions.Timers.Listeners.TimerListener.StartAsync(CancellationToken cancellationToken) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Listener\TimerListener.cs:line 99
   at Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener.StartAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Singleton\SingletonListener.cs:line 70
   at Microsoft.Azure.WebJobs.Host.Listeners.FunctionListener.StartAsync(CancellationToken cancellationToken, Boolean allowRetry) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\FunctionListener.cs:line 68

Now, if we add a connection string to appsettings.json like this:

"ConnectionStrings": {
    "AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=hrintegrationtstsa;AccountKey=**redacted**;EndpointSuffix=core.windows.net"
  },

then the projects starts ok, and the TimerTrigger executes as expected.

The TimerTrigger is in Microsoft.Azure.Webjobs.Extensions and we are at version 4.0.1 (latest)

Can you please comment on our observation? We would like to have this managed identity connection work for TimerTrigger as well.

Kind regards, Chris

[vlkchris]

Hi there - have you been able to look into this yet?

[Ved2806]

Hi @vlkchris Thank you for a feedback, we will investigate this further and let you know about the findings soon.

[Ved2806]

Hi @vlkchris Could you please share a code? how you are trying to configure timerTrigger functionality?

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[vlkchris]

Hi guys - Sorry for the delay from my side. Due to holidays I was not able to comment earlier. I'll provide you with more details tomorrow.

[vlkchris]

Here are some more details of how I am trying to work with TimerTrigger

In Program.cs I have:

var builder = new HostBuilder();
builder.ConfigureWebJobs(b =>
{
    b.AddAzureStorageCoreServices();
    b.AddTimers();
});

I also have a class with a TimerTrigger configured:

namespace WorkerService1
{
    public class TimerProcess
    {
        public void StartByTimer([TimerTrigger("0 * * * * *")] TimerInfo info)
        {

        }
    }
}

In appsettings.json I have:

"AzureWebJobsStorage": {
    "accountName": "hrintegrationtstsa"
  }

Now when I start this project it throws Exception:

info: Host.Startup[0]
      Found the following functions:
      WorkerService1.TimerProcess.StartByTimer

fail: Host.Startup[0]
      The listener for function 'TimerProcess.StartByTimer' was unable to start.
      Microsoft.Azure.WebJobs.Host.Listeners.FunctionListenerException: The listener for function 'TimerProcess.StartByTimer' was unable to start.
       ---> System.ArgumentNullException: Value cannot be null. (Parameter 'connectionString')
         at Microsoft.Azure.Storage.CloudStorageAccount.Parse(String connectionString)
         at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.get_TimerStatusDirectory() in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 73

Now, when I configure a connection string with an account key in appsettings.json

"ConnectionStrings": {
    "AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=hrintegrationtstsa;AccountKey=***redacted***;EndpointSuffix=core.windows.net"
  }

Then the program starts just fine

info: Host.Startup[0]
      Found the following functions:
      WorkerService1.TimerProcess.StartByTimer

info: Function.StartByTimer[1]
      Executing 'TimerProcess.StartByTimer' (Reason='Timer fired at 2022-08-30T16:21:02.5149289+02:00', Id=375c846f-e3e6-44b2-a7e8-d787ac194c97)
info: Function.StartByTimer[0]
      Trigger Details: UnscheduledInvocationReason: IsPastDue, OriginalSchedule: 2022-08-30T16:13:00.0000000+02:00
info: Function.StartByTimer[2]
      Executed 'TimerProcess.StartByTimer' (Succeeded, Id=375c846f-e3e6-44b2-a7e8-d787ac194c97, Duration=22047ms)
info: Host.Triggers.Timer[5]
      The next 5 occurrences of the 'StartByTimer' schedule (Cron: '0 * * * * *') will be:
      08/30/2022 16:22:00+02:00 (08/30/2022 14:22:00Z)
      08/30/2022 16:23:00+02:00 (08/30/2022 14:23:00Z)
      08/30/2022 16:24:00+02:00 (08/30/2022 14:24:00Z)
      08/30/2022 16:25:00+02:00 (08/30/2022 14:25:00Z)
      08/30/2022 16:26:00+02:00 (08/30/2022 14:26:00Z)

Now this connection string contains an account key, but instead of using this account key I want this to work with Managed Identity. When I use this in appsettings.json

"AzureWebJobsStorage": {
    "accountName": "hrintegrationtstsa"
  }

then I can have this app access the storage account using managed identity and that works for BlobTrigger and QueueTrigger. So - the question is why it does not work with TimerTrigger

[marioleed]

Any status on this?

[Ved2806]

Hi @mattchenderson Could you please check with this?

[mattchenderson]

That's super interesting. Do other uses of AzureWebJobsStorage work correctly with identity?

@fabiocav, any ideas? Everything should be using the V12 Storage SDKs, but Microsoft.Azure.Storage.CloudStorageAccount implies an old version still being present somehow.

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[vlkchris]

That's super interesting. Do other uses of AzureWebJobsStorage work correctly with identity?

Yes - I can work with blob and queue triggers using managed identity. TimerTrigger needs old style connection string.

Chris

[mattchenderson]

Thanks. That's very strange - we'll see if we can get to the root of that.

@fabiocav reassigning to you to identify someone on your team to take point here.

[mattchenderson]

One thought - Timer does have that extra abstraction layer for its locking behavior. I wonder if that's the root of it. Maybe the implementation leveraged by WebJobs is defaulting to a different version of that than what we see in Functions, which should have timer working with identity just fine.

[sanshaw79]

Any update on this? I am also facing the same issue.

[franklixuefei]

I verified that with the latest releases, TimerTrigger works with managed identity. You can see that the bp was hit: My nuget dependencies: Here is my setup in appsettings.json:

[kemistry]

@franklixuefei are you getting blobs for timers and locks in your appointed storage?

[mattchenderson]

This should all be resolved now. The packages in question are now GA:

• https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Host.Storage/5.0.0
• https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions/5.0.0
• https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.Timers.Storage/1.0.0

Microsoft.Azure.WebJobs.Extensions.Timers.Storage is a new dependency that you may not have already. It adds back in behavior that was removed from the other packages, hence the major version bump on those. That shuffling allowed the newer SDKs to be brought in in the right places, and the dependencies across theses packages have been cleaned up a bit.

The wire-up is largely the same, but in addition to .AddTimers(), you'll want to call .AddTimersStorage() to set up both the distributed lock manager and the schedule monitor to be backed in Azure Storage.

To set up the managed identity, the config format from Microsoft.Extensions.Azure can be used, reflected in the appsettings.json example above. accountName is non-standard there and won't always work (alternate URLs for different cloud environments), and in general I personally would instead recommend using the service specific URIs:

"AzureWebJobsStorage": {
    "blobServiceUri": "https://<storage_account_name>.blob.core.windows.net",
    "queueServiceUri": "https://<storage_account_name>.queue.core.windows.net",
    "tableServiceUri": "https://<storage_account_name>.table.core.windows.net"
}

We'll close this issue out now that this is supported, and if there are any further issues, please let us know.