azure-docs icon indicating copy to clipboard operation
azure-docs copied to clipboard

Need explanation about container lifecycle for graceful shutdown

Open torumakabe opened this issue 4 years ago • 8 comments

Is there any doc/reference related to container lifecycle on Azure App Service/Functions? I am going to use Functions with docker container for production, so I have to implement graceful shutdown logic. But I cannot find any information about it.

Especially, I would like to understand the behavior when functions stop/restart by Azure portal/API. Does Functions control plane invoke docker stop command? How much time do we have before killing container by Functions control plane? docker default (10 seconds)?

FYI


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

torumakabe avatar Mar 23 '21 02:03 torumakabe

@ToruMakabe Thank you for your feedback! We will review and update as appropriate.

MayankBargali-MSFT avatar Mar 23 '21 03:03 MayankBargali-MSFT

@ToruMakabe If function host crashes, the container is killed immediately. However, you can check https://docs.microsoft.com/en-us/azure/azure-functions/functions-dotnet-class-library?tabs=v2%2Ccmd#cancellation-tokens (for in-process C#)

JayaChatterjee-MSFT avatar Mar 26 '21 16:03 JayaChatterjee-MSFT

@JayaChatterjee-MSFT Thanks. It's an excellent opportunity for improvement. Please could you consider the following from docs point of view?

Clarify the behavior Graceful shutdown is essential for maximizing robustness. If Azure App Service/Functions on docker would not send signals when stop/restart, docs should describe as a limitation. If I understood it in advance, I would not choose these products and not waste time for evaluation. Or consider crash-only design.

Specify the environment in which CancellationToken can be used In my understanding, the IIS in-process hosting model and CancellationToken should only be effective in a limited environment. Maybe only non-docker, Windows/ASP .NET Core model. Many users do not understand the in-process model due to a lack of docs related to the internal design of these products, at least I can not find it. Please explain using terms that are easy for users to understand. (e.g. OS, docker/non-docker, language)

I think these products should send signals before stop/restart, giving feedback to a person in charge also.

torumakabe avatar Mar 27 '21 00:03 torumakabe

@ToruMakabe Thank you for the feedback. @ggailey777 Could you please look into it?

JayaChatterjee-MSFT avatar Mar 29 '21 19:03 JayaChatterjee-MSFT

@JayaChatterjee-MSFT @ggailey777 Any updates? FYI, AWS has published a new excellent article related to this topic. I believe many developers are interested in how to design and implement it.

torumakabe avatar May 24 '21 23:05 torumakabe

Is there any progress on this topic? We have the same issue.

Same issue here. I have a python function which makes a snowflake db connection. I don't want to create connection for each function call. Also, I want to make sure connection is closed before the function app is teared down. I see some capability from python extension but nothing related to tear down task. Would be great if someone provides insight on it.

zahirabbas-v avatar Dec 01 '22 20:12 zahirabbas-v

I got info from the MS Azure team, that this is a bug in Azure, which will be fixed, but it seems that is not fixed yet. I did not get a ETA when it should be fixed.

Our team would really appreciate some documentation on this as well. From some limited testing, it seems like App Service attempts to gracefully shutdown the container, and waits about 5 seconds before killing the container. It's difficult to depend on this though since it's not actually a documented behavior.

Also any kind of configuration on this shutdown timeout would also be really helpful.

goelze avatar Feb 15 '23 19:02 goelze