convoy icon indicating copy to clipboard operation
convoy copied to clipboard

Published 20 hours ago • verified publisher icon rancher

Convoy daemon surprisingly created 12 additional volumes

Open ndelitski opened this issue 10 years ago • 5 comments

Just run sudo convoy create vol-rancher-cattle and got many ebs volumes created in 10 mins. Killed host, stated new one and issue occured again... If volume name contains dots additional volume named . will be created... Convoy 0.3.0 Daemon in EBS mode daemon logs:

time="2015-10-11T18:47:54Z" level=debug msg="Handle plugin volume path: POST /VolumeDriver.Path" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg= event=mountpoint object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:47:54Z" level=debug msg= event=mountpoint mountpoint= object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:47:54Z" level=debug msg="Volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) is mounted at  for docker" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Handle plugin volume path: POST /VolumeDriver.Path" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg= event=mountpoint object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:47:54Z" level=debug msg= event=mountpoint mountpoint= object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:47:54Z" level=debug msg="Volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) is mounted at  for docker" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Handle plugin mount volume: POST /VolumeDriver.Mount" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg="Mount volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) for docker" pkg=daemon 
time="2015-10-11T18:47:54Z" level=debug msg= event=mount object=volume opts=map[MountPoint:] pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:47:54Z" level=debug msg="Volume 5c22798f-7532-4902-93eb-2e3b3190d340 is not mounted, mount it now to /var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340" pkg=ebs 
time="2015-10-11T18:47:54Z" level=debug msg= event=list mountpoint="/var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340" object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:47:54Z" level=debug msg="Response:  {\n\t\"Mountpoint\": \"/var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340\"\n}" pkg=daemon 
time="2015-10-11T18:48:04Z" level=debug msg="Handle plugin unmount volume: POST /VolumeDriver.Unmount" pkg=daemon 
time="2015-10-11T18:48:04Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:48:04Z" level=debug msg="Unmount volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) for docker" pkg=daemon 
time="2015-10-11T18:48:04Z" level=debug msg= event=umount object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:48:04Z" level=debug msg= event=umount object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:48:04Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:48:17Z" level=debug msg="Handle plugin mount volume: POST /VolumeDriver.Mount" pkg=daemon 
time="2015-10-11T18:48:17Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:48:17Z" level=debug msg="Mount volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) for docker" pkg=daemon 
time="2015-10-11T18:48:17Z" level=debug msg= event=mount object=volume opts=map[MountPoint:] pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:48:18Z" level=debug msg="Volume 5c22798f-7532-4902-93eb-2e3b3190d340 is not mounted, mount it now to /var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340" pkg=ebs 
time="2015-10-11T18:48:18Z" level=debug msg= event=list mountpoint="/var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340" object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:48:18Z" level=debug msg="Response:  {\n\t\"Mountpoint\": \"/var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340\"\n}" pkg=daemon 
time="2015-10-11T18:53:09Z" level=debug msg="Handle plugin unmount volume: POST /VolumeDriver.Unmount" pkg=daemon 
time="2015-10-11T18:53:09Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:53:09Z" level=debug msg="Unmount volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) for docker" pkg=daemon 
time="2015-10-11T18:53:09Z" level=debug msg= event=umount object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:53:09Z" level=debug msg= event=umount object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:53:09Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Handle plugin create volume: POST /VolumeDriver.Create" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Found volume 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) for docker" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Handle plugin volume path: POST /VolumeDriver.Path" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg= event=mountpoint object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:27Z" level=debug msg= event=mountpoint mountpoint= object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:27Z" level=debug msg="Volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) is mounted at  for docker" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Handle plugin create volume: POST /VolumeDriver.Create" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Request from docker: &{b71009af882f0bc126ed4dec3d7fd5d128178934476a74dbd5071c01b96e9c58 map[]}" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg="Create a new volume b71009af882f0bc126ed4dec3d7fd5d128178934476a74dbd5071c01b96e9c58 for docker" pkg=daemon 
time="2015-10-11T18:54:27Z" level=debug msg= event=create object=volume opts=map[BackupURL: VolumeName:b71009af882f0bc126ed4dec3d7fd5d128178934476a74dbd5071c01b96e9c58 VolumeDriverID: VolumeType: VolumeIOPS:0 Size:0] pkg=daemon reason=prepare volume=250668c7-15f4-4e18-82e9-2f4f312ed381 volume_name=b71009af882f0bc126ed4dec3d7fd5d128178934476a74dbd5071c01b96e9c58 
time="2015-10-11T18:54:32Z" level=debug msg="Waiting for volume vol-4de8ecbd state transiting from creating to available" pkg=ebs 
time="2015-10-11T18:54:37Z" level=debug msg="Adding tags for vol-4de8ecbd, as map[Name:b71009af882f0bc126ed4dec3d7fd5d128178934476a74dbd5071c01b96e9c58 ConvoyVolumeUUID:250668c7-15f4-4e18-82e9-2f4f312ed381]" pkg=ebs 
time="2015-10-11T18:54:37Z" level=debug msg="Created volume 250668c7-15f4-4e18-82e9-2f4f312ed381 from EBS volume vol-4de8ecbd" pkg=ebs 
time="2015-10-11T18:54:37Z" level=debug msg="Response:  {\n\t\"Err\": \"Cannot find an available device for instance i-189904a1\"\n}" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Handle plugin volume path: POST /VolumeDriver.Path" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg= event=mountpoint object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:49Z" level=debug msg= event=mountpoint mountpoint= object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:49Z" level=debug msg="Volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) is mounted at  for docker" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Handle plugin volume path: POST /VolumeDriver.Path" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg= event=mountpoint object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:49Z" level=debug msg= event=mountpoint mountpoint= object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:49Z" level=debug msg="Volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) is mounted at  for docker" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Response:  {}" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Handle plugin mount volume: POST /VolumeDriver.Mount" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg="Mount volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) for docker" pkg=daemon 
time="2015-10-11T18:54:49Z" level=debug msg= event=mount object=volume opts=map[MountPoint:] pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:49Z" level=debug msg="Volume 5c22798f-7532-4902-93eb-2e3b3190d340 is not mounted, mount it now to /var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340" pkg=ebs 
time="2015-10-11T18:54:49Z" level=debug msg= event=list mountpoint="/var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340" object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:54:49Z" level=debug msg="Response:  {\n\t\"Mountpoint\": \"/var/lib/convoy/ebs/mounts/5c22798f-7532-4902-93eb-2e3b3190d340\"\n}" pkg=daemon 
time="2015-10-11T18:56:47Z" level=debug msg="Handle plugin unmount volume: POST /VolumeDriver.Unmount" pkg=daemon 
time="2015-10-11T18:56:47Z" level=debug msg="Request from docker: &{vol-rancher-cattle map[]}" pkg=daemon 
time="2015-10-11T18:56:47Z" level=debug msg="Unmount volume: 5c22798f-7532-4902-93eb-2e3b3190d340 (name vol-rancher-cattle) for docker" pkg=daemon 
time="2015-10-11T18:56:47Z" level=debug msg= event=umount object=volume pkg=daemon reason=prepare volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:56:47Z" level=debug msg= event=umount object=volume pkg=daemon reason=complete volume=5c22798f-7532-4902-93eb-2e3b3190d340 
time="2015-10-11T18:56:47Z" level=debug msg="Response:  {}" pkg=daemon

fdisk -l

Disk /dev/xvda: 75.2 GB, 75161927680 bytes
255 heads, 63 sectors/track, 9137 cylinders, total 146800640 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *       16065   146785904    73384920   83  Linux

Disk /dev/xvdb: 4289 MB, 4289200128 bytes
255 heads, 63 sectors/track, 521 cylinders, total 8377344 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdb doesn't contain a valid partition table

Disk /dev/mapper/docker-202:1-526147-pool: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders, total 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 65536 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/docker-202:1-526147-pool doesn't contain a valid partition table

Disk /dev/xvdn: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdn doesn't contain a valid partition table

Disk /dev/xvdj: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdj doesn't contain a valid partition table

Disk /dev/xvdi: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdi doesn't contain a valid partition table

Disk /dev/xvdh: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdh doesn't contain a valid partition table

Disk /dev/xvdk: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdk doesn't contain a valid partition table

Disk /dev/xvdl: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdl doesn't contain a valid partition table

Disk /dev/xvdo: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdo doesn't contain a valid partition table

Disk /dev/xvdm: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdm doesn't contain a valid partition table

Disk /dev/xvdp: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdp doesn't contain a valid partition table

Disk /dev/xvdf: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdf doesn't contain a valid partition table

Disk /dev/xvdg: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdg doesn't contain a valid partition table

ndelitski avatar Oct 11 '15 19:10 ndelitski

Hi @ndelitski

In fact from the log, these request comes from Docker, rather than Convoy.

You see e.g.:

time="2015-10-11T18:54:27Z" level=debug msg="Handle plugin create volume: POST /VolumeDriver.Create" pkg=daemon

This kind of request are from Docker. So it's not trigger by your Convoy command.

I found upon the Docker daemon start, it try to find all the existing volumes with current driver(which you may change to Convoy), and ask it to create the volume if they're not there. You can see Handle plugin create volume: POST /VolumeDriver.Create in the log, which means it's coming from Docker(Handle plugin xxx would come from Docker).

yasker avatar Oct 12 '15 18:10 yasker

@ndelitski

And I am unable to reproduce you mentioned "If volume name contains dots additional volume named . will be created..." I tried the middle or end of name, nothing unusual happens... Could you show example of command?

Thanks!

yasker avatar Oct 12 '15 18:10 yasker

Hi @yasker You mentioned that docker demon asks a volume driver to create new volumes if they does not exist. I wonder why convoy created new volumes with random names(11 or 12 - maximum that can be attached to EC2 host) and not complaining about volume not exists? As for dots i just noticed in AWS Console a volume with name '.' and thought it might be related to the name. If i encounter this error i again i will come with more details. In a case above i attached volumes with names vol-rancher.cattle, vol-conf.redis, vol-conf-redis. Docker 1.8.2

AMI ID ubuntu-trusty-14.04-amd64-server-20150325 (ami-47a23a30)

ubuntu ~> docker version
Client:
 Version:      1.8.2
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   0a8c2e3
 Built:        Wed Oct  7 17:48:28 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.2
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   0a8c2e3
 Built:        Wed Oct  7 17:48:28 UTC 2015
 OS/Arch:      linux/amd64

ubuntu ~> docker info
Containers: 8
Images: 174
Storage Driver: devicemapper
 Pool Name: docker-202:1-526147-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 5.728 GB
 Data Space Total: 107.4 GB
 Data Space Available: 66.46 GB
 Metadata Space Used: 9.839 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.138 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.77 (2012-10-15)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-48-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 1
Total Memory: 3.676 GiB
Name: ip-172-30-1-166
ID: S3VY:3PTC:TIGU:ICDC:YFXG:NFS4:ZZXO:CICW:NXXB:VKRP:M6O2:HI4R
WARNING: No swap limit support

daemon init script

ndelitski avatar Oct 12 '15 20:10 ndelitski

@ndelitski

In fact Docker send out create volume request and didn't check if volume exists or not. It would expect driver to provide the volume, identified by the name, and didn't provide information that it would expect volume exists or not. So there is no way for Convoy to know whether Docker want to create a new volume or use a existing volume, thus it would only comply with the request and create the volume.

I guess the name is not random. The volume names should come from your old Docker containers. You can run docker inspect <container_name> to see how container's volume allocated and what's the name of it. And probably you still have the volumes with the same name at /var/lib/docker/volumes locally.

yasker avatar Oct 12 '15 20:10 yasker

I am seeing this in the following scenario:

  • Docker Compose 1.6.2
  • Docker 1.10.3
  • Empty Docker Compose environment
  • Docker Compose includes local volume definitions (v2 specification)
  • When issuing a docker-compose down -v, Convoy creates 4GB volumes for each of the "local" volumes.

I think this is due to Convoy not responding to the POST /VolumeDriver.Get request from Docker:

time="2016-05-03T12:26:28Z" level=debug msg="Handle plugin activate: POST /Plugin.Activate" pkg=daemon
time="2016-05-03T12:26:28Z" level=debug msg="Response:  {\n\t\"Implements\": [\n\t\t\"VolumeDriver\"\n\t]\n}" pkg=daemon
time="2016-05-03T12:26:28Z" level=error msg="Handler not found: POST /VolumeDriver.Get" pkg=daemon
time="2016-05-03T12:26:28Z" level=error msg="Handler not found: POST /VolumeDriver.Get" pkg=daemon
time="2016-05-03T12:26:28Z" level=debug msg="Handle plugin create volume: POST /VolumeDriver.Create" pkg=daemon
time="2016-05-03T12:26:28Z" level=debug msg="Request from docker: &{local_volume map[]}" pkg=daemon
time="2016-05-03T12:26:28Z" level=debug msg="Create a new volume local_volume for docker" pkg=daemon

mixja avatar May 03 '16 12:05 mixja