containers-roadmap [Fargate] [request]: Allow privileged mode

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request I want to be able to mount filesystems within my containers, and for that I need to be able to use privileged mode or add capabilities to the docker container.

Which service(s) is this request for? Fargate

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? Specifically, I am trying to mount an overlay in a container. This requires the container be run in privileged mode.

Are you currently working around this issue? No, there does not appear to be a workaround.

Jul 29 '20 22:07 biltongza

We are interested in this since we are using Gitlab Kubernetes Runners for our CI/CD pipelines and using Docker-in-Docker to build application containers and push them to ECR.

Currently we have larger ec2 nodes that are tainted to only run Gitlab runner pods to avoid resource contention with other applications/service on the EKS clusters. These nodes are only used when CI/CD jobs are in us, and are idle for periods of time. Autoscaling them is also a challenge when many jobs are running at once since cold-starts of EC2 instances can take 2-3 minutes to join the cluster.

By using Fargate Profiles, we would like to setup the gitlab namespace so that any job pods created there will run on their own Fargate node instead of a static ec2 node. This would allow us greater resource flexibility, cost optimization, and separation of jobs to avoid resource contention.

We can do some of our jobs in Fargate like this, but a large majority of our jobs are using docker and docker-compose to build images and run CI/CD tests and would require privileged mode and volume sharing of the Docker socket.

Oct 23 '20 00:10 ecliptik

All the steps are dockerized in my company, so there is no way to use fargate as the CI/CD agent. We are really looking forward to have this ability to run docker in docker on the Fargate platform.

Oct 29 '20 14:10 tata9001

Bump so this not being forgotten. I need to get priv mode working on our current AWS EKS Fargate CI/CD GitLab PoC.

Jan 25 '21 16:01 Waples

+1 For EKS Fargate. We'd like more capabilities to be supported (specifically SYS_PTRACE, DAC_READ_SEARCH) to a container's securityContext. So far it looks like these work:

        securityContext:
          capabilities:
            drop:
              - all
            add:
              - SETPCAP
              - SETUID
              - SETGID

There's an open request for SYS_PTRACE: https://github.com/aws/containers-roadmap/issues/1102.

Jan 27 '21 17:01 sandan

+1 - Open Distro requires Privileged containers to run on EKS.

Apr 07 '21 21:04 jonrau1

s3fs requires privileged also to sync folders:

https://github.com/s3fs-fuse/s3fs-fuse/issues/1246

Apr 20 '21 07:04 FANMixco

+1 for supporting privileged or dockerSecurityOptions . I want to run Chromium in Fargate, but privileged or dockerSecurityOptions is required to enable Chromium's app sandbox.

Jun 09 '21 22:06 binarynate

+1

Aug 08 '21 16:08 amunhoz

Required for any container using s3fs-fuse.

Sep 15 '21 21:09 bradynotarize

Looking at running OpenVSCode in Fargate but need this to be able to run containers inside so docker-in-docker. https://github.com/gitpod-io/openvscode-server

Sep 29 '21 01:09 andymac4182

I need the privileged mode on fargate with ECS so that I can install and run a docker daemon. At this time, docker daemon cannot be started unless the task is being run with the --privileged mode. Please add this capability as soon as possible.

Jan 19 '22 18:01 mayurvin

+1 for this.

Feb 17 '22 13:02 nielsvanoosterom-varias

I would like to be able to use docker-forticlient as a base image for a Fargate task so that my task can reach private resources behind a VPN without a 24/7 IPSec tunnel on a VPC.

https://github.com/HybirdCorp/docker-forticlient

Feb 26 '22 14:02 jarredkenny

Need this for DinD to run Github self hosted runners on Fargate

Apr 19 '22 17:04 askkhan84

+1 to use nsjail inside Fargate

Apr 21 '22 00:04 tekknolagi

Any updates on this? I've tried using buildah, podman, rootless, and nerdctl on fargate but none of them work.

Alternative is to bring up a separate docker build host and expose it over TCP, but that has major operational and security concerns.

Apr 27 '22 19:04 ecliptik

Any updates on this? I've tried using buildah, podman, rootless, and nerdctl on fargate but none of them work.

Alternative is to bring up a separate docker build host and expose it over TCP, but that has major operational and security concerns.

Can this help? Would Kaniko be a viable alternative for you? https://aws.amazon.com/blogs/containers/building-container-images-on-amazon-ecs-on-aws-fargate/

Apr 28 '22 10:04 mreferre

I think the main use case for this request is not to build containers with Fargate, but to be able to mount a file system via FUSE in fargate task. From the first comment:

I want to be able to mount filesystems within my containers, and for that I need to be able to use privileged mode or add capabilities to the docker container.

Apr 28 '22 13:04 pditommaso

Fair enough. I am not sure I'd characterize that as the "main use case" but definitely it was the initial theme of this issue and we should respect that (even though people added additional reasons for needing privileged access, including builds, so it diluted a bit the initial intent).

Apr 28 '22 14:04 mreferre

Would Kaniko be a viable alternative for you?

Kaniko is a no-go for us as it adds additional complexity into our Gitlab CI/CD pipelines.

I think the main use case for this request is not to build containers with Fargate, but to be able to mount a file system via FUSE in fargate task. From the first comment:

Our main request is to build container images using Fargate. We've raised this with AWS Enterprise Support as well, specifically for running Gitlab Runners in Fargate and building and pushing container images to ECR.

Apr 28 '22 14:04 ecliptik

Looking at running OpenVSCode in Fargate but need this to be able to run containers inside so docker-in-docker. https://github.com/gitpod-io/openvscode-server

@andymac4182 Were you able to find a solution for this? If yes, could you please share your approach.

May 06 '22 07:05 anjalichaudhary

+1

May 12 '22 16:05 dasanjaneyuludarla

@anjalichaudhary Ended up just using Gitpod online. Would of preferred Fargate but it doesn't work :(

Jun 08 '22 23:06 andymac4182

+1 for supporting privileged or dockerSecurityOptions . I want to run Chromium in Fargate, but privileged or dockerSecurityOptions is required to enable Chromium's app sandbox.

Did you find any solution? I'm having the same problem.

Aug 08 '22 13:08 nicodp-leap

@nicodp-leap I haven't found a solution, so I'm running Chromium on EC2 instead of Fargate. I would still love to be able to switch it over to Fargate, though.

Aug 08 '22 15:08 binarynate

@binarynate @nicodp-leap one option is to use AWS Batch with EC2 instances in the background. You must set min vCPU to 0 when configuring "compute environments". That will lunch an EC2 instance, run the job in privileged mode and finish the instance after that. You can even use EC2 spot instances for that.