workflow icon indicating copy to clipboard operation
workflow copied to clipboard

Secret Env Not available, container couldn't start

Open pixeleet opened this issue 8 years ago • 4 comments


25m       15h       9         api-cmd-4281948355-qecb1   Pod       spec.containers{api-cmd}   Normal    Pulling      {kubelet ip-10-0-10-170.eu-central-1.compute.internal}   pulling image "bannerwise/core:production-735dd428f31808e52bd422e6e54f5d78bddde289"
25m       15h       9         api-cmd-4281948355-qecb1   Pod       spec.containers{api-cmd}   Normal    Pulled       {kubelet ip-10-0-10-170.eu-central-1.compute.internal}   Successfully pulled image "bannerwise/core:production-735dd428f31808e52bd422e6e54f5d78bddde289"
25m       31m       8         api-cmd-4281948355-qecb1   Pod                                  Warning   FailedSync   {kubelet ip-10-0-10-170.eu-central-1.compute.internal}   Error syncing pod, skipping: failed to "StartContainer" for "api-cmd" with RunContainerError: "GenerateRunContainerOptions: secrets \"api-v115-env\" not found"

What happens is, one of my containers died, a new machine which hasn’t pulled the image yet tried to pull but it somehow didn’t have the latest env secret and thus the container could not start, eventually this resulted in all api containers being not available, still investigating why but if you know something please let me know :stuck_out_tongue: It’s an annoying little thing happening every now and then and I till can’t put my finger on it

Kube version

Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.4", GitCommit:"dd6b458ef8dbf24aff55795baa68f83383c9b3a9", GitTreeState:"clean", BuildDate:"2016-08-01T16:38:31Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}

Deis version

v2.4.0

Cloud Provider: AWS 4.7.3-coreos-r2

pixeleet avatar Feb 14 '17 09:02 pixeleet

@pixeleet can you please try upgrading to something more recent like v2.13.0? There have been plenty of significant changes/bugfixes in the last 11 months and we haven't seen this in recent versions. I'm inclined to think this is long past, and I'd try upgrading first.

bacongobbler avatar Mar 28 '17 19:03 bacongobbler

Sorry this ticket fell through the cracks, by the way. We do appreciate the bug report!

bacongobbler avatar Mar 28 '17 19:03 bacongobbler

Sure, already put upgrading to the latest and greatest deis on the roadmap. It's rarely occuring anyway. On Tue, 28 Mar 2017 at 21:59, Matthew Fisher [email protected] wrote:

Sorry this ticket fell through the cracks, by the way. We do appreciate the bug report!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/deis/workflow/issues/736#issuecomment-289886394, or mute the thread https://github.com/notifications/unsubscribe-auth/AFgmZVbKSWOogw8xNTbhEHutTIJdI5vzks5rqWaMgaJpZM4MAOfX .

pixeleet avatar Mar 28 '17 20:03 pixeleet

This issue was moved to teamhephy/workflow#37

Cryptophobia avatar Mar 20 '18 20:03 Cryptophobia