[Proposal] Add additional logic in docker compose to adapt EFS access to non-root users

[mreferre]

This is to report both a product and a documentation enhancement.

I am trying to deploy this docker compose file to Amazon ECS and I am not able to deploy it “as-is”.

The omero-server container keeps dying due to a permission denied when reading/writing the volume. This is because the omero-server container uses a non-root user id (1000:997). The compose/ECS integration creates an EFS volume and an EFS Access Point on the root directory (/) that only allows the root user to read from / write to. The docker docs that talks about the volumes hint about this and they suggest you could use a specific user to match the POSIX configuration of the file system. This user is basically used to mask the user of the incoming request. In fact, one way to make it work would be to set the mask to use 0:0 (the root user) in the compose file like this:

  omero:
    driver_opts:
      # Access point configuration
      uid: 0
      gid: 0

This, behind the scenes, converts to the following CFN code:

  OmeroAccessPoint:
    Properties:
      AccessPointTags:
      - Key: com.docker.compose.project
        Value: docker-example-omero
      - Key: com.docker.compose.volume
        Value: omero
      - Key: Name
        Value: docker-example-omero_omero
      FileSystemId: fs-7cb2e17b
      PosixUser:
        Gid: "0"
        Uid: "0"
    Type: AWS::EFS::AccessPoint

Another alternative to make this work would be to set a distinct root_directory as the Access Point owned by a random user (e.g. 10000:10000) and setting the mask to this same random user. Like this:

  omero:
    driver_opts:
      # Access point configuration
      uid: 10000
      gid: 10000
      root_directory: “/omero” 
      permissions: 777

This converts to the following CFN code:

  OmeroAccessPoint:
    Properties:
      AccessPointTags:
      - Key: com.docker.compose.project
        Value: docker-example-omero
      - Key: com.docker.compose.volume
        Value: omero
      - Key: Name
        Value: docker-example-omero_omero
      FileSystemId:
        Ref: OmeroFilesystem
      PosixUser:
        Gid: "10000"
        Uid: "10000"
      RootDirectory:
        CreationInfo:
          OwnerGid: "10000"
          OwnerUid: "10000"
          Permissions: "777"
        Path: /omero
    Type: AWS::EFS::AccessPoint

This is basically saying EFS to create a fs, create a folder owned by 10000 with permissions 777, create an AP on that folder and finally masking all requests as user 10000. The documentation doesn't seem to include this example and it would be nice to update the docs to include the syntax for specifying root_directory and permissions (in addition to just add user/group).

Both these two solutions work but they are not ideal because I’d need to tweak the compose to make it work (I would love the compose command to use good judgement and make sensible choices for me instead).

I am wondering if we could embed logic into the docker binary itself so that it is possible, in the case I am creating a volume to be consumed by a container with a non-root user, to make it work without asking the user to tweak their existing compose file.

The Access Point directory could be a random name picked by docker compose or it could be the name of the directory the container would want to mount it to (but not sure if we want to go down the rat-hole of having to deal with nested directories e.g. /my/mount/point).

[ndeloof]

This volume permission issue is comparable to https://github.com/kubernetes/kubernetes/issues/2630. EFS exposing NFS binding option to translate uid/gid offers a workaround, but this still don't reproduce the docker volume experience, which created volume owned by container's user.

I don't think trying to solve this with more hacks would be a good option, especially considering this would only work for new volumes, but not for existing ones.

[mreferre]

Yes I think the workflows for mapping an existing EFS volume should be taken into account here.

In my view if you know that you need to attach an existing volume you are already aware that you need to use the proper driver options. BTW where would I go to read all the driver options I have available? We have some in the examples but it's not the full list as far as I can tell. For example, can I specify the AP in addition to the file system if I want to mount a specific Access Point?

I am wondering if it makes sense to provide a different behavior for newly created volumes? The compose example in my first comment would be the right one. Would it make sense, ONLY for newly created volumes, to either one of the two workarounds (perhaps the first would be the simplest)? If we don't do this we are assuming that people with no AWS knowledge and with an existing super simple docker compose file (like this one) would need to understand the inner details of how EFS works.

[mreferre]

Leaving it here for additional tracking/information purpose only. I was playing with the standard postgres image and I noted I could easily use an EFS volume without having to specify any volume options. The image does use a non-root user:

# explicitly set user/group IDs
RUN set -eux; \
	groupadd -r postgres --gid=999; \
# https://salsa.debian.org/postgresql/postgresql-common/blob/997d842ee744687d99a2b2d95c1083a2615c79e8/debian/postgresql-common.postinst#L32-35
	useradd -r -g postgres --uid=999 --home-dir=/var/lib/postgresql --shell=/bin/bash postgres; \

However it does change ownership and access of the /var/lib/postgresql/data folder:

ENV PGDATA /var/lib/postgresql/data
# this 777 will be replaced by 700 at runtime (allows semi-arbitrary "--user" values)
RUN mkdir -p "$PGDATA" && chown -R postgres:postgres "$PGDATA" && chmod 777 "$PGDATA"
VOLUME /var/lib/postgresql/data