Advice for Functions with 2 to 3 minute runtimes

[TehWardy]

I'm looking for some guidance on the Azure Functions technology stack as I have a feeling that what i'm doing is ok, it's a sub optimal way to use the technology and it's causing me problems ...

Essentially I have a functions app that gets given some details of some data in the back end and then goes and fetches that. It compiles some code and then executes it.

Think of this like a mini workflow engine designed to do some small tasks like transform data from format A to format B. I can't hard code the work in to separate functions but it changes all the time and it's different for each of our clients who are typically very difficult on this front.

So I've found that this results in my functions running about 2 to 3 minutes and I like the "sand boxed nature" of functions as it gives me a degree of control to allow clients to edit this stuff in a UI I've built for them.

So my question(s) here ... Am I using the right technology for this? How might I use it better?

... I've hit a problem that Azure support don't seem to be able to suggest solutions to other than "scale up", in that functions are sometimes just "killed" by the host even on premium plans.

I figure this is because of the relatively "long" run times of my functions. Is there perhaps a functions "pattern" for basically executing a background task or something that I could use instead of just running the code in the function?

I realize that background processes are a thing but I've avoided scheduling the work on a background process / thread and then returning thinking the Host with then kill the functions app instance as it's "done responding".

What am I missing here?

[TehWardy]

I've been looking at the Azure WebJobs SDK stuff here ... https://learn.microsoft.com/en-us/azure/app-service/webjobs-create

Can I host and trigger one of these from an Azure function by just adding the SDK in to my Existing Functions app? Seems like an obvious good choice of tech for longer running operations but I don't know how a functions host instance might behave after I return without awaiting the job to finish.

[bhagyshricompany]

while Azure Functions offer flexibility and scalability, they may not be the best fit for long-running, dynamically generated tasks. Evaluate your requirements, consider alternative services, and explore asynchronous patterns and optimization techniques to improve the performance and scalability of your solution.

[bhagyshricompany]

yes you can use SDK in to my Existing Functions app. Azure WebJobs SDK into your Azure Functions app, you can leverage the strengths of both technologies to build scalable, efficient, and flexible solutions for your workload needs. Remember to design your application logic considering asynchronous processing and potential dependencies between Azure Functions and WebJobs functions.

[TehWardy]

The key problem as I see it is "trust" ... the code in question comes from our clients through our web UI so we don't really want to run that inside a process that's connected to our database (for obvious reasons).

So I thought Azure functions is ideal for this as I can deploy a functions app that "spins up an instance per call" then is thrown away after the processing.

What I would like is a thing that's "a sandbox like an azure function" but still allows me to execute stuff for more than a minute without the unknown headache of it "possibly being killed".

I had considered using something like docker containers but that approach would involve me having to "spin up a container instance" for each call and then tear it down once complete, the overheads of all that "development wise" felt overkill for something that I figured that the cloud should have a built in solution for.

It's like i'm in a weird gap between technologies.

Random question ... Is it possible to just deploy a WebJob with a HttpTrigger that just lives in its own space like an Azure function?

I couldn't find a way, they seem to need a whole app service like a regular web app which doesn't behave the same way as a functions host that is essentially "spawning" a child process in which the code is processed.

[TehWardy]

Hi @brettsam My Azure support engineer suggested reaching out you by name as a key person in this space ...

I was talking to Azure support about the problem I've been facing "Host kills my code randomly" which was the actual underlying problem from what I "thought" was a logging issue.

I would see that my Azure is function is running then it would simply stop, no explanation given no pattern or seemingly common line or particular "situation" in my code that led to the problem the function simply stopped.

Locally I can hit the function all day every day and not hit any problems with it, but when deployed to the cloud I start to get this problem, and the only conclusion we came to was the length of time m functions executions were taking place for.

This led to my chain of thought above ^ ... am I really using the right technology for the problem and if not what IS the right technology?

It gets worse ... When this happens, the deployed functions app seems to "forget" it's configuration state.

I haven't gotten to the bottom of this and I can't debug it since I don't have an exception / any form of stack trace. Whats worse is that Azure Support only had "that's odd, I can't see why this happens either".

I'm left thinking that there's basically only 2 reasons that could be the problem ...

• Roslyn calls do weird things in Azure Functions
• There's some strange time limits that are not well defined (e.g. waiting on something for more than about 90 seconds causes the host to start behaving oddly).

I'd like to talk in more detail on this with someone from the dev team on this issue (teams call if anyone is willing) as it seems like a strange edge case i'm hitting.