portainer
Large image sizes on slow-ish connection cause stack deployment failure notification even though stack deploys
Before you start please confirm the following.
- [x] Yes, I've searched similar issues on GitHub.
- [x] Yes, I've checked whether this issue is covered in the Portainer documentation or knowledge base.
Problem Description
When updating a stack some of the container images might be large (>500 MB). When this is the case a lot of the times the timeout of the "Update the stack" process is not long enough and will produce a "failure to update the stack" notification even though the download is still ongoing, the deployment is still happening and the stack will eventually perform fine.
Expected Behavior
The "update the stack" button triggers a process that runs until either the stack has been fully deployed again or breaks with a more telling error message. Once the stack is updated, the editor quits to show the stack overview.
Actual Behavior
The stack update "fails" because the timeout is not long enough and the user is notified of a "failure". The stack editor stays opened and shows the "Update stack" button like nothing happened. Then you have to leave the stack definition editor, confirming that you want to discard the changes, only to discover that the stack has actually been updated (or will be eventually) while you're doing something else. The error notification should be tied to an actual failure not just a timeout.
Steps to Reproduce
- Deploy a stack or container with a large-ish footprint (>500MB)
- Have a non-ideal internet connection (I have a DSL 150-180Mbit/s real performance line)
- Trigger a "stack update"
- Wait
- Observe the "Failure to update the stack" notification
- Wait a little bit longer
- Check the containers and images to see that, indeed, the new stuff was downloaded and started
- Quit the stack editor anyway
- Observe the running stack
Portainer logs or screenshots
No response
Portainer version
2.34.0
Portainer Edition
Community Edition (CE)
Platform and Version
Docker
OS and Architecture
Synology DSM 7.2
Browser
Firefox 143.0.2
What command did you use to deploy Portainer?
-
Additional Information
Sometimes this behavior also leads to the image being pulled but the stack failing to update. Simply clicking "Update the stack" again will re-trigger the process successfully. This shouldn't be necessary. Either it fails or it succeeds, it shouldn't fail "half way", notify me that it didn't work and then let me make up the difference by clicking again. This isn't intuitive or reliable.
Latest example: stirling-pdf container image fat. image size is 1.32GB, my line would need at least a full minute to download that image alone under ideal conditions. The timeout for waiting for the download definitely isn't that long.
Are you running via a reverse proxy? If so, make sure your timeouts are not cutting the connection
