azure-webjobs-sdk-extensions icon indicating copy to clipboard operation
azure-webjobs-sdk-extensions copied to clipboard

Support Azure Files

Open rustd opened this issue 8 years ago • 62 comments

Add a new trigger binding for Azure Files. Here is some more info https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-files/

rustd avatar Oct 05 '15 18:10 rustd

We'd want both a trigger as well as input/output bindings.

mathewc avatar Oct 27 '16 05:10 mathewc

We definitely need this FileTrigger, or a way to mount a drive in a WebJob and use the FileTrigger from WebJob.Extensions. Currently this approach is possible from within an Azure Servcie as it runs as NetworkService, but impossible in a WebJob as it runs as IIS APPPOOL\your_hosting_web_app, which has no network access. Trying to change the identity of the pool using the advanced settings and a applicationHost.xdt with the content shown below does not work either, transform seems to be fully ignored.

This is all really very sad, as such a trigger would make hybrid solutions with customers having no IT personnel trivial: anybody can drop a file in a share...

Hoping to have a solution for this real quick.

<?xml version="1.0"?> <configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform"> <system.applicationHost> <applicationPools> <add name="%XDT_SITENAME%"> <processModel identityType="NetworkService" xdt:Transform="SetAttributes(identityType)"/> </add> </applicationPools> </system.applicationHost> </configuration>

mliesmons avatar Dec 28 '16 08:12 mliesmons

Please do this. This would enable so many lift and ship scenarios.

g4mb10r avatar Jan 03 '18 01:01 g4mb10r

+1

andersborum avatar Mar 14 '18 20:03 andersborum

This work would likely happen after V2 GAs, but this would be a great feature.

/cc @jeffhollan

fabiocav avatar Mar 14 '18 21:03 fabiocav

I'd also like to know when a file is deleted. When the researchers remove a file, we need to remove it from the DB.

MisinformedDNA avatar Apr 12 '18 16:04 MisinformedDNA

need file-share trigger

koolkarni avatar May 10 '18 18:05 koolkarni

Any updates on this feature? Or is there a plausible workaround while we wait for a feature release?

jhossy avatar Aug 23 '18 11:08 jhossy

No updates - our team doesn't have the time to work on this right now, so the feature request remains in backlog. Note we already have a Files extension that works for the regular file system - this work item is about supporting Azure Files.

mathewc avatar Aug 24 '18 17:08 mathewc

@mathewc Are you referring to external files, which only work in V1 and is marked as experimental?

MisinformedDNA avatar Aug 30 '18 13:08 MisinformedDNA

I created a library that monitors changes in the local file system or on a network drive. Since Azure File Share supports SMB, maybe it can be applied here? You would just set up a recurring job to get the latest changes and then either immediately process them or send them to an Event Grid topic to act as the trigger.

I haven't had time to play with this case, but I think it would work. If anyone wants to give it a go, let me know if it works. Otherwise, I'll try to flesh out the details.

UPDATE: The only way my code would work is if we span up a VM just to host the SMB connection. That's a non-starter for me.

MisinformedDNA avatar Aug 30 '18 13:08 MisinformedDNA

@MisinformedDNA the only missing piece I see with that is you don't know who changed the file.

For audit logging/HIPAA compliance I would like events for Read/Write/Delete. I submitted a feedback item here: https://feedback.azure.com/forums/217298-storage/suggestions/35896822-events-for-azure-files

evictor avatar Nov 02 '18 20:11 evictor

Any update on this?

a-patel avatar Dec 05 '18 13:12 a-patel

@fabiocav and @jeffhollan - Any hope here? Thanks.

silverl avatar Dec 19 '18 16:12 silverl

Up?

guillaume-kizilian avatar Feb 08 '19 09:02 guillaume-kizilian

Nearly 4 years on. Any update whatsoever on this guys?

GFoley83 avatar May 12 '19 04:05 GFoley83

The path forward to adding new trigger support for Azure Functions is having the service integrate with Azure Event Grid. We have made this request to the Azure Storage team and continue to do so, but the big item here that would enable this would be Azure Files supporting Azure Event Grid for 1st party events around file operations.

jeffhollan avatar May 14 '19 18:05 jeffhollan

Functions can trigger on Blob storage events (https://docs.microsoft.com/en-us/azure/azure-functions/functions-triggers-bindings) and many other bindings. How long until Storage Account/Files can be supported?

glittle avatar May 16 '19 17:05 glittle

How can we get the Storage team involved in this discussion? Or is there a better place to have this discussion?

MisinformedDNA avatar May 16 '19 18:05 MisinformedDNA

@MisinformedDNA I just added a forum question here: [https://social.msdn.microsoft.com/Forums/azure/en-US/ed7fed62-fcc5-42b1-bae3-0f2850c4cb5e/add-a-trigger-binding-for-azure-files?forum=AzureFunctions](MSDN Forums)

glittle avatar May 16 '19 19:05 glittle

@glittle Probably the wrong forum as you chose Azure Functions instead of Azure Storage. (Also, your link is reversed.)

MisinformedDNA avatar May 16 '19 19:05 MisinformedDNA

Triggers are defined in Functions, so wouldn't that be where to ask?

glittle avatar May 16 '19 19:05 glittle

At first glance, you might think so, but as @jeffhollan has said above

The path forward to adding new trigger support for Azure Functions is having the service integrate with Azure Event Grid. We have made this request to the Azure Storage team and continue to do so, but the big item here that would enable this would be Azure Files supporting Azure Event Grid for 1st party events around file operations.

Basically, Azure Functions doesn't want to create triggers, they want to work off of triggers. And, really, that's how it should be. We don't need Azure Functions polling our file share like crazy when the Storage team can just know when something happens and raise an event.

MisinformedDNA avatar May 16 '19 19:05 MisinformedDNA

Will that do away with Function triggers?

Since Blobs are files, I may be able to do what I need using Blob Storage triggers.

glittle avatar May 16 '19 19:05 glittle

Blobs work fine.

MisinformedDNA avatar May 16 '19 19:05 MisinformedDNA

The issue I have with using Blobs is the warning here: MS Warning In addition, storage logs are created on a "best effort" basis. There's no guarantee that all events are captured. Under some conditions, logs may be missed.

ltdavids avatar May 23 '19 19:05 ltdavids

BlobTrigger currently employs a hybrid strategy - it uses blob log scan to get notified of changes, but it also does a background receipt based scan of the actual blobs as well. So even if a blob log is missed, the blob will eventually get processed.

mathewc avatar May 23 '19 20:05 mathewc

For those of us who work in the IT services industry, using a service that has no guarantee of working is a deal-breaker for SLAs.

GFoley83 avatar Jul 02 '19 18:07 GFoley83

Any update here?

ak09 avatar Mar 12 '20 19:03 ak09

I too would appreciate this feature.

JoePyx avatar May 12 '20 17:05 JoePyx