docker-elastic-agents-plugin icon indicating copy to clipboard operation
docker-elastic-agents-plugin copied to clipboard

Container not removed if creation fails

Open chrisb2244 opened this issue 1 year ago • 0 comments

When running GoCD with the Docker Elastic Agent Plugin (v3.0.0-245), if the creation of the container fails (example log below) then the container is left in a "Created" state, and not removed.

The container below is a Windows image, based on windows/servercore. Connection is via named pipe (npipe:////./pipe/dockerDesktopWindowsEngine is the value for "docker_uri" in the cluster configuration).

[go] Received request to create a container of <image> at 2022-08-26 10:54:01 +09:00
[go] Creating container: dc43e776-28f4-414f-ae1d-d82787358004
[go] Starting container: dc43e776-28f4-414f-ae1d-d82787358004
[go] Failed while creating container: Request error: POST npipe://localhost:80/containers/dc43e776-28f4-414f-ae1d-d82787358004/start: 500, body: {"message":"hcsshim::CreateComputeSystem 3237419004cf2143e4a23c3606fe0eeb52038d03ad27ea71ed0850317ff83ca1: Insufficient system resources exist to complete the requested service."}
[go] Received request to create a container of<image> at 2022-08-26 10:55:38 +09:00
[go] The number of containers currently running is currently at the maximum permissible limit, "12". Not creating more containers for jobs: blah1, blah2, ... blah6.
[go] Received request to create a container of <image> at 2022-08-26 10:57:37 +09:00
[go] The number of containers currently running is currently at the maximum permissible limit, "12". Not creating more containers for jobs: blah1, blah2.
[go] Received request to create a container of <image> at 2022-08-26 10:59:37 +09:00
[go] Creating container: 7ebdf5aa-caa6-4505-921f-def26500d4fe
[go] Starting container: 7ebdf5aa-caa6-4505-921f-def26500d4fe
[go] Started container: 7ebdf5aa-caa6-4505-921f-def26500d4fe
[go] Job Started: 2022-08-26 11:00:06 JST

In this case, container 7ebdf5aa stops and is cleaned up correctly, but some time later, the following is the output of docker ps --all:

CONTAINER ID   IMAGE      COMMAND                  CREATED          STATUS      PORTS        NAMES
3237419004cf   <image>    "powershell -command…"   14 minutes ago   Created                  dc43e776-28f4-414f-ae1d-d82787358004
aa0d180ffe74   <image>    "powershell -command…"   14 minutes ago   Created                  2fe6d7cd-b5f6-4349-ae9a-3b206dd25c8f
8ce302b782e1   <image>    "powershell -command…"   14 minutes ago   Created                  ec2cfa54-d37a-49af-8a95-84ba3ac45fcd
291c53086ec3   <image>    "powershell -command…"   14 minutes ago   Created                  ef526633-ada9-4128-9ee7-20c1b301cbb0
535ec6df68b1   <image>    "powershell -command…"   15 minutes ago   Created                  81709ac4-beed-467f-8feb-9ef04b12f3c9

dc43e776 didn't get cleaned up (I guess because it never started) and needs to be manually removed with docker rm <name/id> or docker container prune.

Can these be tracked by the plugin and removed automatically?

chrisb2244 avatar Aug 26 '22 02:08 chrisb2244