Docker ECS integration fails on EFS mount with multiple named volumes

[bounlu]

I am using Docker ECS integration to deploy my multi-container docker compose application as described here: https://docs.docker.com/cloud/ecs-integration/

I use these commands:

docker context create ecs aws
docker context use aws
docker compose -f Dockerfile.aws.yml up

Docker uses volumes mapping to Amazon EFS together with AWS Fargate. I have multiple volumes (n = 10) per container to be mounted on EFS, but I get error due to the 1 mount target per AZ limitation as described here: https://docs.aws.amazon.com/efs/latest/ug/API_CreateMountTarget.html

The error message is like this: ⠼ myapp DeleteInProgress The following resource(s) failed to create: [InputsNFSMountTargetOnSubnetbc5to1da...

Resource handler returned message: "Rate exceeded (Service: Efs, Status Code: 400, Request ID: f12cf024-91ae-4a06-96d5-fe40a128228b, Extended Request ID: null)" (RequestToken: 231b2732-bec3-f98b-ae2d-5b6510a2d84f, HandlerErrorCode: GeneralServiceException)

I have only 3 AZ in my region ap-southeast-1. So I can deploy a maximum of 3 volumes. How to fix this?

My Dockerfile.aws.yml file:

version: "3.9"

services:
  web:
    image: myrepo/web
    x-aws-pull_credentials: arn:aws:secretsmanager:ap-southeast-1:039873776982:secret:hahaha-23TcxA
    depends_on:
      - database
    ports:
      - 80:80
    volumes:
      - results:/var/www/html/myapp/results
      - library:/var/www/html/myapp/library
      - software:/var/www/html/myapp/software
      - inputs:/var/www/html/myapp/submit/inputs
      - sequences:/var/www/html/myapp/upload/sequences
      - files:/var/www/html/myapp/upload/uploads/files
      - files_fastqc:/var/www/html/myapp/upload/uploads/files_fastqc
      - files_trimlog:/var/www/html/myapp/upload/uploads/files_trimlog
      - files_description:/var/www/html/myapp/upload/uploads/files_description

  database:
    image: myrepo/database
    x-aws-pull_credentials: arn:aws:secretsmanager:ap-southeast-1:039873776982:secret:hahaha-23TcxA
    volumes:
      - db:/var/lib/mysql

volumes:
  db:
  results:
  library:
  software:
  inputs:
  sequences:
  files:
  files_fastqc:
  files_trimlog:
  files_description:

[minenwerfer]

Hi, have you found a workaround for this already?

[bounlu]

No, I couldn't.

[robbie1douglas]

I'm experiencing the same issue with trying to get a Graylog PoC up on ECS following the same example.

[JohnPreston]

I am surprised that the ECS integration considers all these volumes as EFS.

Without explicitly saying that these volumes should be on EFS via setting the driver_opts feels ... weird.

Out of curiosity would each docker volume be a different filesystem ? Would not having a single filesystem and different access points (https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html) work ? :thinking:

Are you trying to have 1 EFS Filesystem per volume or a same filesystem for each of these volumes but different mount points?

I am asking to replicate in ECS Compose-X volumes. IMO, these volumes defined in volumes should just be shared volumes across the containers of the same task definition, not EFS by default. Got plenty of use-cases for that.

Curious about the use-case to see on how to implement in ecs-compose-x :thinking:

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it had not recent activity during the stale period.