docker: Error response from daemon: chown /var/lib/docker-volumes/netshare/nfs/testvolume: operation not permitted.

[xang555]

trafficstars

hi everyone i need your help !

i had some error when i'm run docker container. here my command for run docker container :

docker-volume-netshare nfs

create volume

docker volume create -d nfs --name testvolume -o share=my_nfsserver_ip:/var/data

finally i'm run container by command

docker run -i -t -v testvolume :/var/lib/ghost --name myblog -P ghost

and i have error message :

docker: Error response from daemon: chown /var/lib/docker-volumes/netshare/nfs/test: operation not permitted.

## how to fix it ?

thank for your help.!

[gondor]

Can you add the logs outputted from docker-volume-netshare. Also what version of docker and what OS are you running this from?

[xang555]

ok here

docker-volume-netshare log :

time="2017-01-06T18:52:56+07:00" level=info msg="== docker-volume-netshare :: Version: 0.18 - Built: 2016-05-27T20:14:07-07:00 =="
time="2017-01-06T18:52:56+07:00" level=info msg="Starting NFS Version 4 :: options: ''"
time="2017-01-07T16:39:08+07:00" level=info msg="== docker-volume-netshare :: Version: 0.18 - Built: 2016-05-27T20:14:07-07:00 =="
time="2017-01-07T16:39:08+07:00" level=info msg="Starting NFS Version 4 :: options: ''"
time="2017-01-07T16:45:13+07:00" level=info msg="Mounting NFS volume 103.208.24.41:/var/data on /var/lib/docker-volumes/netshare/nfs/myvol"
time="2017-01-07T16:45:13+07:00" level=info msg="Unmounting volume name myvol from /var/lib/docker-volumes/netshare/nfs/myvol"
time="2017-01-07T16:47:20+07:00" level=info msg="Mounting NFS volume 103.208.24.41:/var/data on /var/lib/docker-volumes/netshare/nfs/myvol"
time="2017-01-07T16:47:20+07:00" level=info msg="Unmounting volume name myvol from /var/lib/docker-volumes/netshare/nfs/myvol"
time="2017-01-07T16:53:49+07:00" level=info msg="Mounting NFS volume 103.208.24.41:/var/data on /var/lib/docker-volumes/netshare/nfs/myvol"
time="2017-01-07T16:53:49+07:00" level=info msg="Unmounting volume name myvol from /var/lib/docker-volumes/netshare/nfs/myvol"
time="2017-01-07T16:54:34+07:00" level=info msg="== docker-volume-netshare :: Version: 0.18 - Built: 2016-05-27T20:14:07-07:00 =="
time="2017-01-07T16:54:34+07:00" level=info msg="Starting NFS Version 4 :: options: ''"

docker version :`

Docker version 1.12.5, build 7392c3b

OS is ubuntu 16.04.1 x64 :

Linux bro2-vm 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[gondor]

Your running quite an old version of docker-volume-netshare. Have you tried with a newer binary? There have been many bug fixes since then and the version you're running was pre docker 1.12

[xang555]

@gondor i remove old version of docker-volume-netshare and install new version 0.33 .

ho No!!!. i had error again when i run container

like this :

docker: Error response from daemon: create 103.208.x.x/var/data: create 103.208.x.x/var/data: Error looking up volume plugin nfs: plugin not found.

i had start service docker-volume-netshare and run service docker-volume-netshare nfs

how to do that ?.

thank you.

[josselinchevalay]

hi @gondor

i have same issue than @xang555.

docker-volume-netshare :: Version: 0.33 - Built: 2017-01-08T22:45:48-08:00

Client: Version: 1.12.3 API version: 1.24 Go version: go1.6.3 Git commit: 34a2ead Built: OS/Arch: linux/amd64

Server: Version: 1.12.3 API version: 1.24 Go version: go1.6.3 Git commit: 34a2ead Built: OS/Arch: linux/amd64

however i use coreos so i create an systemd jobs with that command :

./docker-volume-netshare nfs

[josselinchevalay]

hi all,

i created a new VM to test :

• OS : SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux
• docker version :

Client:
 Version:      1.12.5
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   7392c3b
 Built:        Fri Dec 16 02:21:54 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.5
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   7392c3b
 Built:        Fri Dec 16 02:21:54 2016
 OS/Arch:      linux/amd64

• docker info :

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 1.12.5
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 1
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local nfs
 Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options:
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 3.824 GiB
Name: joss
ID: IKJM:FH5C:2TYT:HWCR:Y5UW:XBZP:OWIQ:AHVY:7HMM:OBWA:7T5Z:7AA3
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: http://proxy.ullink.lan:9876/
Https Proxy: http://proxy.ullink.lan:9876/
No Proxy: ulcentral.ullink.lan:5001
Registry: https://index.docker.io/v1/
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
Insecure Registries:
 127.0.0.0/8

log

root@joss:/home/joss# docker-volume-netshare nfs 
INFO[0000] == docker-volume-netshare :: Version: '0.20' - Built: '2016-08-28T20:15:48Z' == 
INFO[0000] Starting NFS Version 4 :: options: ''        
INFO[0010] Mounting NFS volume 192.168.0.161:/vol/backup_nfs_srm on /var/lib/docker-volumes/netshare/nfs/192.168.0.161/vol/backup_nfs_srm 
2017/01/10 10:01:46 mount.nfs4: Protocol not supported

INFO[0010] Unmounting volume name 192.168.0.161/vol/backup_nfs_srm from /var/lib/docker-volumes/netshare/nfs/192.168.0.161/vol/backup_nfs_srm 
2017/01/10 10:01:47 umount: /var/lib/docker-volumes/netshare/nfs/192.168.0.161/vol/backup_nfs_srm: not mounted

so i need to i need compare my install under coreos and debian.

how to change nfs version in your project i need to use oldest version.

i mount that with

sudo mount -t nfs <my_host>:<my-path> <my_target>

Regards

[josselinchevalay]

hi,

i checked gap under my coreos and debian VM. Under coreos i haven't nfs.sock in /run/docker/plugins

any idea

[josselinchevalay]

hi,

to coreos i fund an solution how to install plugin : https://docs.docker.com/engine/extend/plugin_api/

[TJC]

Hello -- I think I might be experiencing the same problem as the original poster.
Or maybe not..

I'm running version 0.33 of the netshare plugin, with CIFS. The daemon is indeed running correctly, its logs look fine.

However the Docker daemon throws an error when I try to use a cifs volume:

#!/bin/bash
docker volume create -d cifs --name inbox \
  -o share=server.my.domain.name/inbox \
  -o username=inbox \
  -o password=passwordpassword \
  -o domain=MYDOMAIN

docker run -ti --rm -v inbox:/srv ubuntu /bin/bash

The output is: docker: Error response from daemon: chown /var/lib/docker-volumes/netshare/cifs/inbox: permission denied.

[TJC]

The docker.service logs read:

`Jan 16 17:58:50 adonai dockerd[32680]: time="2017-01-16T17:58:50.804402005+11:00" level=error msg="Handler for POST /v1.24/containers/create returned error: chown /var/lib/docker-volumes/netshare/cifs/inbox: permission denied"

[rkrzewski]

I also run into this problem, and after spending significant time of this I was able to find a workaround.

I've observed that NFS volume was mounted successfully into the container only when the mount point directory DID NOT exist inside the containers image. When the image had the mount point directory, even if it was empty and uid:gid values of the mount point inside the container were perfectly aligned with the uid:gid of the exported directory on the NFS server, starting the container would fail with:

docker: Error response from daemon: chown /var/lib/docker-volumes/netshare/<share>: permission denied.

It turns out that when a new named docker volume is created (and as I understand plugin-provided volumes are named by default) docker attempts to adjust the ownership and permission of the target directory to match the image and extract data from the image into that directory. Those operations are peformed under uid of docker daemon ie root, unless --userns-remap is used.

If the NFS server has root_squash option on (which is a sensible default) the uid 0 (root) on client side is mapped to 65534 (nobody) and AFAICT this is the reason of the chown error above.

Luckily, this behavior can be disabled using nocopy option when mounting the volume to container:

docker run -d --volume-driver nfs -v nfs_server:/exports/data:/data:nocopy the_image

Will mount the volume correctly even when the_image contains a /data directory.

If you need to extract data from your image, you must use a host directory volume first, mount the NFS share manually and copy the files over, tear down the container and finally recreate it using NFS share volume with nocopy mode on.

[jgato]

@rkrzewski thank so much for this great point!!! I have working in this issue during a week. My problem is trying to use Mongo with an NFS shared volume. Now, using nocopy option the container starts and mount the nfs, but the mounted directory cannot be touched. The process exists because it tries to write in the shared directory /data/db and it cannot.

I have entered into the container, even as root I cannot write in this directory. Any help about that?

[jonaskello]

Maybe my comment and solution in #85 will help you.