for-aws icon indicating copy to clipboard operation
for-aws copied to clipboard

Published 20 hours ago • verified publisher icon docker-archive

Moby Linux 17.12.ce: Locked out after the instance is restarted.

Open dperetti opened this issue 7 years ago • 9 comments

Expected behavior

After a restart from the EC2 management console, it should be possible to log in again. # Welcome to Docker !

Actual behavior

After a restart from the EC2 management console, it's no longer possible to log in. Forever. docker@<instance-ip>: Permission denied (publickey,keyboard-interactive).

Steps to reproduce the behavior

  1. Launch an EC2 instance of ami-7abe2e03 (17.12.0-ce-aws1).
  2. ssh docker@<instance-ip>
  3. # Welcome to Docker !
  4. Restart the instance from the EC2 management console.
  5. ssh docker@<instance-ip>
  6. # docker@<instance-ip>: Permission denied (publickey,keyboard-interactive).

Please note that the issue does not arise using ami-38ad2d41 (17.09.1).

dperetti avatar Feb 01 '18 14:02 dperetti

@dperetti Can you provide logs from the ssh container, so that we can identify what's possibly causing this? Can you also add the instance boot logs?

FrenchBen avatar Feb 02 '18 10:02 FrenchBen

Indeed, there is an interesting part during the second launch!

Loaded image: docker4x/shell-aws:17.12.0-ce-aws1
 [ ok ]
 * Setup SSH ...sshkey
docker: Error response from daemon: Conflict. The container name "/shell-aws" is already in use by container "58ef529bd8087656fa68240a53bc67c4d883c25ca519c435549e8b65f2f7310d". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.

system-log.txt

dperetti avatar Feb 02 '18 12:02 dperetti

Until there is a fix in place, we've worked around this by restarting the ssh container in User Data.

docker container restart shell-aws

ph-One avatar Feb 02 '18 14:02 ph-One

@ph-One thanks for the help, I need a bit of help identifying why this is happening - the shell container is created with -d --restart=always.

@dperetti this doesn't "hurt" anything, as it's simply stating that you cannot create a new container with the same name. The existing container is still there and running.

FrenchBen avatar Feb 02 '18 16:02 FrenchBen

Through a bit of hammering I was able to narrow it down to the LinuxKit dockerd - See issue created here: https://github.com/moby/moby/issues/36189

nb: I was able to replicate the same issue with a simple nginx container on Docker for Mac

FrenchBen avatar Feb 02 '18 16:02 FrenchBen

Well now - quite a bug!

ccamp46 avatar Feb 02 '18 19:02 ccamp46

I can confirm this is still happening with the following AWS AMI: Moby Linux 18.03.1-ce-aws1 stable (ami-8ac542f5).

The fix mentioned by @ph-One still works, but seems to be far from ideal. Any ideas when this might get resolved?

sachnk avatar May 22 '18 04:05 sachnk

Same Moby Linux 18.03.0-ce-aws1 stable (ami-3129f94c)

sshilko avatar Jun 22 '18 08:06 sshilko

Still happening in the following:

  • Moby Linux 18.03.1-ce-aws1 stable - ami-8ac542f5

niboowin avatar Aug 05 '18 22:08 niboowin