dinghy-http-proxy
dinghy-http-proxy copied to clipboard
codekitchen
Resolving in containers without dinghy
Hi,
I switched from dinghy to Docker on mac and use now your very good proxy / dns container.
While it's working well from host to container, I a unable to resolve *.docker inside containers.
Do you have some advice on how to achieve this ?
Thanks,
Hm, that sounds like Docker for Mac is not using the host DNS resolver. It seemed to be the last time I tried it, but that was a few months ago.
Will need to do some investigation, and see if there's even a way to change Docker for Mac's behavior here.
I was experiencing this issue too - Docker for Mac doesn't resolve anything in /etc/resolver for some reason.
@jessepollak I resolved it by adding an additional Dnsmasq container with a static IP and configuring my containers with the dns option.
I've confirmed this doesn't work anymore with Docker for Mac version 1.12.0-a. As far as I've been able to figure out, the VM is running its own DNS server and the docker daemon is configured to use this server, so it's writing that configuration to each container's /etc/resolv.conf. I don't see any configuration options around this setup, unfortunately.
We could modify the dnsmasq process running in dinghy-http-proxy to be a full dns server, that continues to resolve *.docker but forwards on all other hostnames to the upstream dns resolvers. But we still need the containers to use that resolver. As @mickaelperrin suggests, you can add the --dns option to each docker run command to tell the container to use this resolver. That's a lot of work though, ideally we'd add the --dns option to the docker daemon itself. I'm not seeing a way to modify docker daemon options with Docker for Mac so far, though. This might be worth opening an issue against that project too.
Another thing we could try is to actually modify the OS X DNS settings to add our resolver. If we do that, the DNS server that Docker for Mac is running might use the resolver too. I'm not positive of that but it might be worth a shot.
It looks like this is a known issue with Docker for Mac: https://forums.docker.com/t/docker-pull-not-using-correct-dns-server-when-private-registry-on-vpn/11117/8
They have a private bug tracker where this is assigned issue # 3124, but it doesn't sound like we have any way to track the issue or see updates on it. 😞
@codekitchen It looks like the latest docker for mac release allows to provide daemon configuration. Do you have any idea on which dns IP or other settings should I provide to resolve this issue ?
I created a loopback alias on the host with the command sudo ifconfig lo0 alias 10.2.3.4. It allows me to route traffic from container to the host (mainly used for debuggers).
Should I set 10.2.3.4 ?
Can't test at the moment, I did a full docker reset and I am currently redownloading all my base images on my so sloooow DSL connection.
Since the DNS resolver is running in a container in the VM, listening on port 19322, you should be able to just set the IP to 127.0.0.1 and configure to use that port. I haven't tried it yet, let me know if that works for you.
Did anyone get this working? I'm still having issues getting containers to resolve other containers. Thanks!
I haven't had a chance to try out that potential fix yet, I need to re-install docker for mac.
Sadly, it looks like there is no way to specify a port in the server configuration. cf. https://github.com/moby/moby/issues/6924 and https://github.com/moby/moby/issues/6905
Darn... I suppose we could try exposing DNS on the standard port 53 instead with -p 53:19322/udp but I don't know how likely that is to conflict with other things.
Has there been any progress on this? I'm wanting to switch from dinghy to Docker for Mac, but I haven't been able to get DNS resolution inside the containers working.
Correction: I thought I had this fully working, but following the below steps fixes in-container resolving but breaks host resolving.
Thanks for reminding me about this open issue. I do have this working, but it relies on using an internal IP address that Docker for Mac generates. So far it seems stable in my usage, but I'm not sure how often, if ever, this IP might change.
Basically you just follow the README instructions, but the VM_IP for docker run isn't 127.0.0.1. You need to find out what IP host.docker.internal resolves to (for me, it resolves to 192.168.65.2 but I'm not sure if that's always the case). An easy way to do this is to just run docker run --rm bash ping host.docker.internal.
So in my case, I run:
docker run -d --restart=always \
-v /var/run/docker.sock:/tmp/docker.sock:ro \
-v ~/.dinghy/certs:/etc/nginx/certs \
-p 80:80 -p 443:443 -p 19322:19322/udp \
-e DNS_IP=192.168.65.2 -e CONTAINER_NAME=http-proxy \
--name http-proxy \
codekitchen/dinghy-http-proxy
And my /etc/resolver/docker is just:
nameserver 127.0.0.1
port 19322
If 127.0.0.1 doesn't work in the resolver file, I would try using that same host.docker.internal IP, but the latest versions of Docker for Mac seem to support Mac OS resolver files again.
I also get 192.168.65.2 on the couple of Macs I've tried. However, I can also confirm that in-container resolving works but host-to-container resolving is broken:
docker run -d --restart=always \
-v /var/run/docker.sock:/tmp/docker.sock:ro \
-v ~/.dinghy/certs:/etc/nginx/certs \
-p 80:80 -p 443:443 -p 19322:19322/udp \
-e DNS_IP=192.168.65.2 -e CONTAINER_NAME=http-proxy \
--name http-proxy \
codekitchen/dinghy-http-proxy
docker run --rm -d --name nginx -e VIRTUAL_HOST=nginx.docker nginx:latest
docker run --rm -d --name nginx -e VIRTUAL_HOST=apache.docker httpd:2.4
docker run --rm byrnedo/alpine-curl http://nginx.docker
... returns nginx default page ...
docker run --rm byrnedo/alpine-curl http://apache.docker
... returns default apache page ...
curl http://apache.docker
.... hangs
From the host mac, the *.docker hostnames are resolving to 192.168.65.2:
dns-sd -G v4 apache.docker
d209-087-058-057:~ dashboard$ dns-sd -G v4 apache.docker
DATE: ---Tue 16 Apr 2019---
12:57:18.458 ...STARTING...
Timestamp A/R Flags if Hostname Address TTL
12:57:18.461 Add 2 0 apache.docker. 192.168.65.2 15
^C
Yeah at first I thought 192.168.65.2 was reachable from the host, but it must've just been caching.
I was hoping to find an IP address that will work from both host and containers, but I haven't had any luck there. The best solution might be to run two DNS servers like @mickaelperrin originally described.