draft-classic icon indicating copy to clipboard operation
draft-classic copied to clipboard

[>.12 draft releases]: Long first remote push kills the release process

Open squillace opened this issue 7 years ago • 7 comments

On Mac, draft version: &version.Version{SemVer:"canary", GitCommit:"78385afe65500ebb4f546341f229a5f200f1128a", GitTreeState:"clean"}

The image shows you what happens again and again IF you've never pushed before. In this case, pushing to docker.io/squillace. If you immediately rerun draft up, it all succeeds fine, and subsequent rebuilds with changes will push much faster.

image

squillace avatar Apr 05 '18 22:04 squillace

This one is interesting. I haven't the foggiest idea what caused it, though it looks like it was a connection timeout error with the port-forwarded connection to tiller. I'll see if I can reproduce. :)

Before that error occurred, were you able to confirm that tiller was indeed running? It's possible that the cluster was still trying to start up.

bacongobbler avatar Apr 05 '18 22:04 bacongobbler

same cluster, every time. Here it happens again, with the first time deploying to the same AKS cluster as above but from ACR. Wondering whether we're maybe trying to deploy before the image is ready to be pulled? No idea; just guessing here.

image

squillace avatar Apr 06 '18 00:04 squillace

Remember: I'm on a home network for which the upload is VASTLY slower than most people's.... might have something to do with it. Look at the original upload times.....

squillace avatar Apr 06 '18 00:04 squillace

Yeah: this is a thing. If the push takes too long -- and the initial push often does on a slow upload connection -- the release step fails on this result. the tunnel context has been lost, I think, but it doesn't reconnect/reconstruct the ssh context.

squillace avatar Apr 17 '18 01:04 squillace

hmm good insight! we establish a connection to tiller the moment we invoke draft up. Perhaps there's a way to keep the connection alive.

bacongobbler avatar Apr 26 '18 15:04 bacongobbler

Or let me set the timeout somewhere..... don't do work FOR me, but in the small number of cases when upload is KNOWN slow, a hard to find timeout setting can be expanded...... ?

squillace avatar Apr 26 '18 18:04 squillace

labeling as a good first issue. :)

bacongobbler avatar May 03 '18 05:05 bacongobbler