Option to Mount Volume at Sub Directory

[quater]

I started to evaluate Volplugin today and really like it. Great work!

Volplugin is a strong contenter for my project and I am wondering whether you would consider the following feature request.

Status Quo

Volplugin allows to mount volumes at their root location as Docker does equally.

$ docker run -it -v policy/drive:/drive:ro ubuntu /bin/bash

The Problem

I would like to be able to mount a volume at a given sub directory position. The reason being is that I want to have one volume that is shared across numerous containers but each container only gets to see data from a particular sub directory or gets files mounted directly.

In my particular use case the volume is used in a micro services setting and holds dynamically generated configuration files for a number of containers/ services. I don't want containers to have access (not even read-only) to configuration files that belong to other containers for security purposes.

At present I can archive this will following hack but I would prefer a cleaner method.

$ docker run -it -v policy/drive:/drive:ro -v /mnt/ceph/rbd/policy1/drive/etc/nginx/conf.d/web.conf:/etc/nginx/conf.d/web.conf:ro ubuntu /bin/bash

Note: The -v policy/drive:/drive:ro option will cause Volplugin to create a local mount on the Docker host at /mnt/ceph/rbd/policy1/drive, which is then used by the second mount option.

Proposed Feature

Ideally I thought we could have something like this...

$ docker run -it -v policy/drive/etc/nginx/conf.d/web.conf:/etc/nginx/conf.d/web.conf:ro ubuntu /bin/bash

..., but that is not possible to implement without Docker's help since they don't allow slashes in the volume name. (see https://github.com/docker/docker/issues/17734)

However the below syntax should be possible to implement.

$ docker run -it -v policy/drive_._etc_._nginx_._conf.d_._web.conf:/etc/nginx/conf.d/web.conf:ro ubuntu /bin/bash

I know this is not overly elegant due to the necessity of signifying the sub directory/ file path by using the _._ annotation but it seems feasible.

I have been poking around and currently see the following two starting points for implementing this. https://github.com/contiv/volplugin/blob/master/volplugin/init.go#L52 https://github.com/contiv/volplugin/blob/master/volplugin/handlers.go#L61

What is your standpoint?

[erikh]

This sounds great. What I'd like to propose instead of doing this with a -v syntax is to do this within the policy; it'll be easier to control at the high-level entry point docker run is supposed to be, and easier to manage from a visibility (what mount is where) perspective.

FWIW, -v policy/test/some/dir would also work, but I'm not sure that's how I'd like this to work, though. Docker itself is not the limitation, I can attest to that. The volume merely must not contain a leading /. With either strategy you would need to docker volume create for each volume sub-path instead of for separate policies. In the policy saves us some general trouble by allowing it to be recorded and examined through the volcli tool. docker volume create --opt path=/my/path can be used to specify this at create time, making it a little easier than filling out json each time you need to do something.

Another significant problem is how storage drivers will handle this.

Thoughts?

Thank you for your interest!

[erikh]

The real trick to this is in the storage driver, which must detect and prevent duplicate mounts for a sub-mount, so only the original mount is managed and the bind mounts are per-container.

This may be complex.

[quater]

The implications and complexities are much greater than I initially through but I feel this feature is worth it as it would make Volplugin the Docker volume extension of choice for so many others.

Here is what I understood from your response in the shape of a sequence of commands that would work if the feature is implemented.

$ volcli policy upload policy1 < policy1.json
$ volcli policy get policy1 | jq '.'

{
  "name": "policy1",
  "unlocked": true,
  "create": {
    "size": "10MB",
    "filesystem": ""
  },
  "runtime": {
    "snapshots": true,
    "snapshot": {
      "frequency": "30m",
      "keep": 20
    },
    "rate-limit": {
      "write-iops": 0,
      "read-iops": 0,
      "write-bps": 0,
      "read-bps": 0
    }
  },
  "driver": {
    "pool": "rbd"
  },
  "filesystems": {
    "ext4": "mkfs.ext4 -m0 %"
  },
  "backends": {
    "crud": "ceph",
    "mount": "ceph",
    "snapshot": "ceph"
  }
}

Note: "unlocked": true so multiple containers across multiple Docker hosts can access the volume.

Up to this point, it's status quo. Now to the new stuff...

1. Creating volume with two submount paths

$ docker volume create -d volplugin --name policy1/foo --opt path=/my/path1
$ docker volume create -d volplugin --name policy1/foo --opt path=/my/path2

or

$ docker volume create -d volplugin --name policy1/foo --opt path=/my/path1 --opt path=/my/path2

or

$ docker volume create -d volplugin --name policy1/foo --opt paths=["/my/path1","/my/path2"]

Note: We only would need either one of the above commands implemented. Whatever is feasible.

This results in the below policy json.

$ volcli volume get policy1/foo

{
  "policy": "policy1",
  "name": "foo",
  "driver": {
    "pool": "rbd"
  },
  "mount": "",
  "create": {
    "size": "10MB",
    "filesystem": "ext4"
  },
  "paths": {
    "0": "/my/path1",
    "1": "/my/path2"
  },
  "runtime": {
    "snapshots": true,
    "snapshot": {
      "frequency": "30m",
      "keep": 20
    },
    "rate-limit": {
      "write-iops": 0,
      "read-iops": 0,
      "write-bps": 0,
      "read-bps": 0
    }
  },
  "backends": {
    "crud": "ceph",
    "mount": "ceph",
    "snapshot": "ceph"
  }
}

Note: The new section is

  "paths": {
    "0": "/my/path1",
    "1": "/my/path2"
  },

Provided that Docker passes through the volume string policy1/foo/my/path1:/whatever/location to volplugin without throwing the error: docker: Error response from daemon: create policy1/foo/my/path1: "policy1/foo/my/path1 includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. the volume and its submounts can be mounted with below commands.

$ docker run -it -v policy1/foo/my/path1:/whatever/location ubuntu bash
$ docker run -it -v policy1/foo/my/path2:/whatever/location ubuntu bash

or for read-only

$ docker run -it -v policy1/foo/my/path1:/whatever/location:ro ubuntu bash
$ docker run -it -v policy1/foo/my/path2:/whatever/location:ro ubuntu bash

This should result in the volume to be mounted on the Docker host only once regardless of how many containers access it through the submounts.

What do you think?

[erikh]

what are the 0,1 here? indexes or something else?

[erikh]

Hmm. I have a better idea that builds on yours.

We keep the paths notion in the policy, but make it editable at docker run -v time. This would work by fooling docker into THINKING the volume exists when it requests to mount it, but not reporting it in the list of volumes. This would provide a really smooth UX I think.

example:

$ docker volume ls
policy1/test
policy1/test2

$ docker run -d -v policy1/test/hello/world:/mnt alpine sh

The mounts would also be cleaned up on container stop which would make it very convenient to use as one-off container things like tools and scripts that need to run against the volume can be run as separate containers.

WDYT? @mapuri @jainvipin @dseevr @unclejack

[quater]

@erikh That sounds good!

[erikh]

Ok I should warn you to be fair: it will take some time to get this done as we're preparing for our first release. After that it will be easier to tackle.

[quater]

Thank you for considering this feature. I will get by for the time being and look forward to your release!