Docker-from-docker: host.docker.internel is not resolving as expected

[Antman261]

• VSCode Version: 1.68.0 (Universal)
• Local OS Version: macOS 12.3
• Local chip architecture: Apple Silicon
• Reproduces in: Codespaces
• Name of Dev Container Definition with Issue: Docker-from-docker

The Problem

When using Codespaces and docker-from-docker, Containers are unable to resolve the host IP address via host.docker.internal as they would normally and per the docs: https://docs.docker.com/desktop/linux/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host

Steps to Reproduce:

I have built a repository that perfectly encapsulates and reproduces this problem:

• https://github.com/Skutopia-org/spike-debug-codespaces-docker-networking

Run ./run.sh

This script will start a "Hello world" node server on the host machine, port 3000. Then the script runs docker compose up, which builds and runs an image executing curl http://host.docker.internal:3000

Locally (Expected outcome)

The docker image can successfully curl the node server via host.docker.interal:3000

Attaching to spike-debug-codespaces-docker-networking-example-service-1
spike-debug-codespaces-docker-networking-example-service-1  |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
spike-debug-codespaces-docker-networking-example-service-1  |                                  Dload  Upload   Total   Spent    Left  Speed
100    11  100    11    0     0   1355      0 --:--:-- --:--:-- --:--:--  1375
spike-debug-codespaces-docker-networking-example-service-1 exited with code 0

Codespaces (Actual outcome)

Using the provided Codespaces config with docker-from-docker, the curl container cannot resolve the host and fails.

The output is as follows:

Attaching to spike-debug-codespaces-docker-networking-example-service-1
spike-debug-codespaces-docker-networking-example-service-1  |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
spike-debug-codespaces-docker-networking-example-service-1  |                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: host.docker.internal
spike-debug-codespaces-docker-networking-example-service-1 exited with code 6

Workaround

Currently, we work around this issue by hardcoding the host IP address, which so far has always been 172.17.0.1

Context

Our real-world use-case is that we are running a Hasura server via docker compose, and it needs to access a node server running on the "host" for development purposes. Locally this works fine, but in Codespaces, we had issues until we discovered the above workaround.

[joshspicer]

Hey! A couple questions for you so I can best understand your use case

1. Is your real-world scenario defined by a docker-compose.yml? If that's the case, have you considered editing your .devcontainer.json to directly run that compose file (see: https://containers.dev/implementors/json_reference/#compose-specific)

2. Have you taken a look at our docker-in-docker script/feature (link), that will create a new isolated docker daemon inside your container (hence the "in"). From a user perspective in Codespaces, this will behave more similarly to how it would locally.

The docker-from-docker script is something that can work in Codespaces, but note that you will then have access to the docker socket that we internally spin up your "Codespaces" container onto (the VM running your codespace is totally isolated to you, and this script will potentially let you kill your container "view" into the Codespace). Just something to note.

[eitsupi]

When using Codespaces and docker-from-docker, Containers are unable to resolve the host IP address via host.docker.internal as they would normally and per the docs: docs.docker.com/desktop/linux/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host

As the documentation says, host.docker.internal only works with Docker Desktop, so I think it is a intended behaver. Please check this issue https://github.com/docker/for-linux/issues/264#issuecomment-964620100.

[Antman261]

In our real-world use case we use docker-compose, but not everything is in docker compose. The main services we develop are ran on the host, primarily an actions server and a react client, both of which we run with hot reloading while developing. We also have a service that watches and builds changes in a symlinked package in the file system, which allows us to share types and runtime parsing between client and server.

I don't think specifying the docker-compose.yml in devcontainer.json is appropriate for our use case because we do not want to connect to a service within that compose. Trying to do so would break significant parts of our development workflow.

It sounds like docker-in-docker would suit our usecase better -- it wasn't clear to me what the differences between these two scripts were, or their implications. We'll give it a try!

[joshspicer]

@Antman261 has docker-in-docker worked for you?

[Antman261]

Yes, it has! 😅

[mego22]

@Antman261, would you mind expounding on your setup? I'm trying to get a similar workflow up in Codespaces but docker-in-docker is not working for me.