`docker-from-docker`: allow installation alongside `docker-in-docker`, as non-default docker configuration

[davidwallacejackson]

#171 prevents me from using docker-from-docker as the primary docker configuration for my devcontainer (since I need to share docker ports with internal processes). I realized, however, that there was nothing preventing me from mounting both sockets and switching off between them. This allowed me to get what I really wanted out of docker-from-docker, which was the ability to update the devcontainer image on the host machine. This is useful for two reasons, both pertaining specifically to locally-running devcontainers:

1. I can use the devcontainer CLI to iterate on builds without shutting down the active dev container -- so if I'm making changes, I don't need to lose access to my project while I wait to see the there are any errors.
2. I can implement a "check for updates" process, where I use the host's docker daemon to check for the newest built version of the devcontainer image, download it, and then encourage the user to Rebuild Container at their earliest convenience if and only if the image has actually changed since the one they're currently using. My image is privately hosted and I can't access my private registry from the bootstrap container (because doing so would require access to the host's credential helpers), so this is the only way to ensure that users start a Rebuild with an up-to-date cache.

I'm accomplishing this right now by vendoring some pieces of the docker-from-docker feature into a custom feature, but it seems like it would make more sense as a toggle for the existing feature: let me use my devcontainer configuration to set the target socket to e.g. /var/run/docker-host-proxy.sock instead of /var/run/docker.sock. Then I can use DOCKER_HOST=/var/run/docker-host-proxy.sock from scripts inside my devcontainer to interact specifically with the host's Docker daemon.

[joshspicer]

What is preventing you using docker-in-docker as your primary means of using docker with dev containers? Do you have containers running on your local machine that you want to use inside the dev container? Could you do all of your work inside of the dev container (docker-in-docker style), or with docker-compose orchestrated by the dev container CLI, where everything would be on the same docker network and could be configured how you'd like it?

I may not totally understand the use case yet, but it seems reasonable to me to allow an override option to configure the path to socket in docker-from-docker

[davidwallacejackson]

@joshspicer I am currently using docker-in-docker as my primary means of running Docker, yeah -- and it works great for almost everything. The reason I want docker-from-docker as well is that I want users to be able to pull the newest build of the devcontainer from the registry from inside the devcontainer while it's running, then do "Rebuild From Container" and have it Just Work™. Of course, if I do that while running docker-in-docker, the pulled build won't be present in the host's docker daemon, so the Rebuild will result in the entire container build running from scratch (which can take 10+ minutes).

To give more context: my CI builds/pushes my devcontainer on every commit for ARM and AMD both, using the devcontainer CLI (I use earlier commits as cache-from, so this is always a trivial operation if it hasn't changed). It then creates a manifest joining the ARM and AMD builds and pushes it under the current commit SHA and the branch name. The devcontainer.json specifies "cacheFrom": ["myPrivateRegistry/devcontainer:master"], so when a user opens the application in a devcontainer, it's built against the most recent master build -- which is nearly guaranteed to be up-to-date with whatever the actual devcontainer config is in the state of the repository at the time, unless they're actively hacking on the devcontainer config.

The problem is that this scheme assumes that the host machine has an up-to-date version of myPrivateRegistry/devcontainer:master. In theory, I could just do a docker pull myPrivateRegistry/devcontainer:master in my initializeCommand, but initializeCommand runs inside of a bootstrap container when you're doing Clone Repository In Container Volume -- and I have to instruct my users to do that because the filesystem performance when I have the whole repo in a bind mount isn't good enough yet. The bootstrap container doesn't have credentials for my private registry, so I can't pull there.

I do, however, ask my users to log into gcloud as part of the standard environment setup instructions, which allows them to access the private registry from their host. And I bind-mount the gcloud credentials into the container so that they're automatically logged in there as well. So inside the devcontainer, where I have a bunch of scripts and automation running already, I have the opportunity to pull the latest devcontainer image -- I can even check if the hash of the image has changed and give the developer a heads up that they should rebuild! But to do all that, I need docker-from-docker -- so I hacked up the thing I have now, with both sockets available, and on the rare occasion I want the host I call it out in my scripts.