Efs volume does not accessible for existing containers after reboot(docker-volume-netshare efs started)

[fyls]

Hi. i have problem with containers which use nfs volumes and should run after host reboot. Is anyone has the same problem with shares after reboot? Thank you for your help.

System info OS - Centos 7.1 (3.10.0-327.22.2.el7.x86_64) Docker - 1.12.0 build 8eab29e GO - go1.6 linux/amd64 docker-volume-netshare - 0.18

HOW IT INSTALLED

INSTALLING EFS UTILS

sudo yum install -y nfs-utils

INSTALL GO

wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz tar -C /usr/local -xzf /tmp/go1.6.linux-amd64.tar.gz ln -s /usr/local/go/bin/go /usr/local/bin/go ln -s /usr/local/go/bin/godoc /usr/local/bin/godoc ln -s /usr/local/go/bin/gofmt /usr/local/bin/gofmt

ADD GO PATH'S

tee ~/.bashrc <<-'EOF' export PATH=$PATH:/usr/local/go/bin export GOROOT=$HOME/go export PATH=$PATH:$GOROOT/bin EOF

INSTALL docker-volume-netshare

go get github.com/ContainX/docker-volume-netshare cd $GOROOT/src/github.com/ContainX/docker-volume-netshare go build

INSTALL DOCKER

sudo tee /etc/yum.repos.d/docker.repo <<-'EOF' [dockerrepo] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/7/ enabled=1 gpgcheck=1 gpgkey=https://yum.dockerproject.org/gpg EOF

Start docker

systemctl start docker

Start docker on startup

systemctl enable docker

add to autoload

sudo tee /etc/systemd/system/dockerefs.service <<-'EOF' [Unit] Description = Load docker plugin for efs After = docker.service

[Service] StartLimitInterval=5 StartLimitBurst=10 User=root ExecStart = /root/go/bin/docker-volume-netshare efs Restart=always RestartSec=120 [Install] WantedBy = multi-user.target EOF

Start dockerefs

systemctl start dockerefs.service

Start dockerefs on startup

systemctl enable dockerefs.service

PROBLEM

• After reboot docker demon can hangs, update service can't heelp. We should manually stop demon and rerun it again. because efs mounts depended it's can't load
• UPD. removed all containers service start normal
• UPD. if we run container with EFS mount as demon with option restart always docker demon hang's on boot, after restart dameon we will get "Error response from daemon: get fs-86a0XXXX: no such volume"
• After reboot even if we have running netshare-efs we got "Error response from daemon: get fs-86a0XXXX: no such volume" when we start container, if we run new container with efs, previous one can start without problems.

HOW TO REPRODUCE docker run -i -t --volume-driver=efs -v fs-86aXXXX:/tmp ubuntu /bin/bash

Starts normal, directory inside mounted, no problem after reboot. (container shouldn't start after reboot automatically) If we start it by hand - we got problem "Error response from demon: get fs-86a0XXXX: no such volume". if we start another container with the save volume and start previous, no problem, and share accessible

docker run -d --volume-driver=efs -v fs-86aXXXX:/tmp --restart always nginx

Starts normal, after reboot docker demon hangs.

[gondor]

Just posted a new release today. One of our contributors has been fixing up the EFS driver and these changes have made it in the 0.20 release today. Since I don't have EFS maybe you can test your cases again using the latest version

[EmilioMoreno]

Hi I do not know if this is the same issue, but looks quite similar, i am running 0.20 under debian jessie. Service is installed and started properly:

docker-volume-netshare.service                                                            loaded active running   LSB: Init for docker-volume-netshare
docker.service                                                                            loaded active running   Docker Application Container Engine

root@Deb8x64vm:/home/dalamar# systemctl status docker-volume-netshare.service
● docker-volume-netshare.service - LSB: Init for docker-volume-netshare
   Loaded: loaded (/etc/init.d/docker-volume-netshare)
   Active: active (running) since Fri 2016-09-02 18:46:26 CEST; 14min ago
  Process: 513 ExecStart=/etc/init.d/docker-volume-netshare start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/docker-volume-netshare.service
           └─535 /usr/bin/docker-volume-netshare nfs

Sep 02 18:46:26 Deb8x64vm docker-volume-netshare[513]: Starting Docker-Volume-Netshare: docker-volume-netshare.

and the container which is using it runs perfectly if i start it manually. However, if i restart the server, when docker try to restart it, it gives the error "no such volume"

Failed to start container bec53d9af0a16c7c3f6c478c4653c713e692189a99619a1a332fe9d18c58345f: get 192.168.100.200: no such volume"

if i try to restart the container manually i get the same error

docker restart btSync
Error response from daemon: Cannot restart container btSync: get 192.168.100.200: no such volume

however if i remove the container, and create it again it works perfectly.

[r5r3]

The problem is reproducible with NFS volumes as well. The docker-volume-netshare.service is started, the docker.serviceis also started, but the NFS volumes are missing. They are not listed by docker volume ls anymore.

Containers are not able to start as their volumes are missing. To restart the containers, it is necessary to manually recreate the NFS volumes.

[manast]

@r5r3 I think this is works as design, i.e., volumes are not recreated automatically when booting in docker.

[PeterBennink]

Is there some way to fix this? I have the problem with the NFS volumes like @r5r3 describes, and for my specific use case it's sort of necessary..

[leialexisjiang]

I have the same problem, all volumes disappear after rebooting and I just don't understand why this is an expected behaviour. it's a bug for me or maybe there are some limitations in docker side. I'm taker if someone have solutions. thx