sail icon indicating copy to clipboard operation
sail copied to clipboard

dind support

Open ammario opened this issue 6 years ago • 23 comments

ammario avatar Apr 15 '19 00:04 ammario

because of the pitfalls outlined here https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ we'll just mount in the docker socket and provide an OUTER_ROOT env with the path to chroot from the host's perspective.

ammario avatar May 02 '19 01:05 ammario

I think it'd also be nice to have this as a label in case the user doesn't want docker in their env

nathanpotter avatar May 02 '19 01:05 nathanpotter

discussed in person that we can always provide the OUTER_ROOT env and leave it to the user to bind mount the socket in with a share directive. Should definitely provide a tutorial on getting this set up.

ammario avatar May 02 '19 01:05 ammario

What are the important pitfalls that make DIND a no go?

nhooyr avatar May 07 '19 20:05 nhooyr

also, vscode remote mounts the path outside the container at the same path inside the container so that mounts work as expected. See https://github.com/Microsoft/vscode-dev-containers/tree/master/containers/docker-in-docker

We could do this too.

nhooyr avatar May 07 '19 20:05 nhooyr

I don't see that in that doc?

ammario avatar May 19 '19 04:05 ammario

Proper dind is supposedly unreliable according to that blog post... Going to close this for now.

ammario avatar May 19 '19 04:05 ammario

Proper dind is supposedly unreliable according to that blog post... Going to close this for now.

Could you expand on this? As far as I can tell, its just edge cases.

nhooyr avatar May 19 '19 15:05 nhooyr

Docker-in-Docker: the bad However, contrary to popular belief, Docker-in-Docker is not 100% made of sparkles, ponies, and unicorns. What I mean here is that there are a few issues to be aware of. One is about LSM (Linux Security Modules) like AppArmor and SELinux: when starting a container, the “inner Docker” might try to apply security profiles that will conflict or confuse the “outer Docker.” This was actually the hardest problem to solve when trying to merge the original implementation of the -privileged flag. My changes worked (and all tests would pass) on my Debian machine and Ubuntu test VMs, but it would crash and burn on Michael Crosby’s machine (which was Fedora if I remember well). I can’t remember the exact cause of the issue, but it might have been because Mike is a wise person who runs with SELINUX=enforce (I was using AppArmor) and my changes didn’t take SELinux profiles into account.

ammario avatar May 19 '19 18:05 ammario

Thats an edge case. Won't affect most users.

nhooyr avatar May 19 '19 18:05 nhooyr

The only other issue I see is the build cache... We're trying to promote short-lived environments so Docker in the container may be slow.

ammario avatar May 19 '19 19:05 ammario

Thats fine imo, I'm sure we can mount in an image cache somehow later. At least it'll work.

nhooyr avatar May 19 '19 19:05 nhooyr

Wouldn't we want to enable dind in our ubuntu-dev base image, forcing every container to be privileged?

ammario avatar May 19 '19 19:05 ammario

Ah yea, that's true. That is very unfortunate.

nhooyr avatar May 19 '19 19:05 nhooyr

Why don't we take vscode's approach?

also, vscode remote mounts the path outside the container at the same path inside the container so that mounts work as expected. See https://github.com/Microsoft/vscode-dev-containers/tree/master/containers/docker-in-docker

We could do this too.

nhooyr avatar May 19 '19 19:05 nhooyr

I never understood what you found novel about their approach. Aren't they just bind mounting in the Docker socket?

ammario avatar May 19 '19 19:05 ammario

Yes but they bind mount the project path in the same place as it is on the host. So you can mount things from your project (which is all most people want) easily.

nhooyr avatar May 19 '19 20:05 nhooyr

So if the project is at /home/ammar/Projects/test on the host it has the same path in the container... Wouldn't we have to overhaul our user system them? Right now the user is always user.

ammario avatar May 19 '19 20:05 ammario

So if the project is at /home/ammar/Projects/test on the host it has the same path in the container... Wouldn't we have to overhaul our user system them? Right now the user is always user.

Not necessarily, we can just mount the project in there, the user inside the container can remain the same.

Also, I don't think running containers privileged is a big deal as users do expect docker to just work and we're not advertising security benefits.

nhooyr avatar May 19 '19 20:05 nhooyr

Just bind mounting in the project is easy enough...

We had a discussion about privileged by default on Slack. You were of the opinion that it's a bad idea then

ammario avatar May 19 '19 21:05 ammario

Also not sure how that bind mount solution helps... If I'm in the container and trying to bind mount my project, I'll probably be using the project directory within /home/user/.

ammario avatar May 19 '19 21:05 ammario

Just bind mounting in the project is easy enough... We had a discussion about privileged by default on Slack. You were of the opinion that it's a bad idea then

I was wrong.

Also not sure how that bind mount solution helps... If I'm in the container and trying to bind mount my project, I'll probably be using the project directory within /home/user/.

But that's the thing, your project won't be mounted into /home/user, it'll be on the host path inside the container. E.g. lets say your project exists at /Users/nhooyr/projects/nhooyr/websocket, it'd get mounted inside the container at /Users/nhooyr/projects/nhooyr/websocket.

Either way, privileged by default is fine imo.

nhooyr avatar May 19 '19 23:05 nhooyr

Privileged by default is also good with me, so I suppose we should implement proper dind and see where that gets us.

ammario avatar May 20 '19 00:05 ammario