devilbox icon indicating copy to clipboard operation
devilbox copied to clipboard

Published 20 hours ago • verified publisher icon cytopia

WIP: Release v1.2.0: Auto DNS (breaking changes)

Open cytopia opened this issue 6 years ago • 11 comments

AutoDNS

READFIRST

This branch is constantly being rebased from master until final merge, therefore requires git force pushing. Keep this in mind when checking it out via git.

Feedback

To move on faster and pain-free, please try out this branch and provide your Feedback. I will update the description correspondingly.

  • [ ] Tested on Linux
  • [ ] Tested on Windows (native)
  • [ ] Tested on MacOS (native)
  • [ ] Tested on Windows (Docker Toolbox)
  • [ ] Tested on MacOS (Docker Toolbox)

DESCRIPTION

This PR splits the currently called DNS container bind into two separate containers: intdns and autodns

  • Refs #248
  • Refs https://github.com/cytopia/devilbox/issues/119

intdns (required to run)

The intdns container will only be responsible to handle DNS queries internally for the PHP container, directing all queries to the Devilbox HTTP server. This container must always run in order to be able to resolve virtual hosts from within the PHP container (e.g. curl https://project1.loc).

autodns (optional)

The autodns container will be responsible to take care about the DNS for your host operating system. Previously all queries resolved to 127.0.0.1 by default. This however did not work for legacy solutions such as Docker Toolbox. In order to mitigate this issue, this container has been made configurable to specify the IP address where the Devilbox is serving the virtual hosts. For native solutions this is set to 127.0.0.1 by default. If you're using Docker Toolbox, you can set this value to 192.168.99.100 via the .env file for instance:

###
### The IP address to which direct all DNS resolutions.
###
### This settings really depend on where you want to access your Devilbox projects
### from your host computer's browser.
###
### If you access the Devilbox on 127.0.0.1, then set this to 127.0.0.1
### If however you're running Docker Toolbox and the Devilbox IP address is something
### like 192.168.99.100, then set it to 192.168.99.100. (Note, this is only an example)
### In case of Docker Toolbox, you will have to find out the IP address first.
###
AUTODNS_HOST_ADDRESS=127.0.0.1

State

This PR is currently WIP, but already usable if you want to try it out. There are however still a few things that need to be addressed before this can be released.

Todo

  • Update documentation (especially how to setup various frameworks and change address of MySQL/PgSQL/etc to their respective hostnames)
  • Update PHP-FPM container and remove port-forwards and make them even slimmer (port-forwards of services to 127.0.0.1 into the PHP container will not be required any more)

Important

Once you're using this feature, you should not rely on any services being bound to 127.0.0.1 on the PHP container anymore. You should from now on either:

  1. stick to their hostnames
  2. or use custom environment variables to tell PHP where other services can be reached: https://devilbox.readthedocs.io/en/latest/configuration-files/env-file.html#custom-variables

The second option is anyhow much better in case you decide for example to use a database outside the Devilbox docker environment. Then you will only have to adjust the .env variable and you're all set.

Benefits

Other benefits will include for easier DNS resolution inside the LAN. (e.g. access the Devilbox from various devices: mobiles, tablets, other computers without having to change a lot, even easier usage as a staging or prod server - if u intend to do so)

cytopia avatar Nov 04 '18 06:11 cytopia

I am testing the autdns branch with:

  • [x] Tested on Windows ( Docker Toolbox with version v18.09.0 )
  • [x] My docker machine IP : 192.168.99.100

I can use Virtual Hosts by editing the C:\Windows\System32\drivers\etc\hosts manually like: 192.168.99.100 website1.loc

With Port Forwarding in the Virtualbox like:

virtualbox-port

I can then call the devilbox with 127.0.0.1 and editing the hosts like: 127.0.0.1 website2.loc works either.

The autodns feature unfortunately has no effect with .env like:

  • HOST_PORT_BIND=53
  • AUTODNS_HOST_ADDRESS=192.168.99.100

This leads to:

  • PHP Container Setup/Settings: DNS Offline
  • PHP Container Statrus/Bind connect: Could not connect to Bind via hostname:

fkaminski avatar Jan 11 '19 13:01 fkaminski

@fkaminski thanks for the heads up. Looks like I will need to invest some work into this one.

cytopia avatar Jan 12 '19 18:01 cytopia

Hi, i am really not familar with the whole network administration stuff at all. I have tried lots of combinations

  • your branches/releases with or without port forwarding
  • VirtualBox Settings like --natdnsproxy1 on or --natdnshostresolver1 on
  • Virtualbox with "Host-only Adpater" or "Bridged Adapter"
  • even tested images like https://hub.docker.com/r/andyshinn/dnsmasq
  • etc.

but did not get the autodns feature to work at all ...

So if you need any further info to get it done with Docker Toolbox - Please let me know ...

fkaminski avatar Jan 14 '19 14:01 fkaminski

Finally get it worked with Docker Toolbox:

A fresh Installation of Docker Toolbox v18.09.0 with "Virtualbox NDIS6 Bridged Networking Driver" ( which is the default installation option ) and Windows 7 ( Windows 10 Home not yet tested ).

After further setup via "Docker Quickstart Terminal" you get a "default" VM with "Network Adapter 1" as "NAT" and "Network Adapter 2" as "Host-only Adapter" per default in Virtualbox.

My Docker Machine IP is: 192.168.99.100 ( docker-machine ip )

Then: In the Windows Network Konfiguration do NOT set the Preferred DNS IP as 127.0.0.1 in the "normal/active" network adapter ( as for now described in the devibox Dokumentation ) - Instead:

Set the docker machine IP ( here 192.168.99.100 ) as preferred DNS IP in the "Virtualbox Host-Only Network #2" Adapter - see Screen:

win_netwok_setting 0

After that i restart my computer and do a "ipconfig /flushdns" in the Windows CMD - just for shure.

Then with GitBash in the devilbox folder:

  • git fetch --all --prune
  • git checkout autodns

and the following entries in the .env:

  • LOCAL_LISTEN_ADDR=
  • HOST_PORT_BIND=53
  • AUTODNS_HOST_ADDRESS=192.168.99.100

Then:

  • docker-machine start # Or start via VirtualBox GUI
  • docker-compose up -d autodns httpd php

With Chrome Browser head to http://192.168.99.100: Although i get the following info in the devilbox admin:

devilbox_admin 0

the virtual host can be reached:

devilbox_admin_vhost 0

Furthermore: If we setup port forwarding in the Virtualbox "NAT Adapter" like:

virtualbox_port_forward 0

we can also browse http://127.0.0.1 for the devilbox web interface and the virtual hosts work as well ...

fkaminski avatar Jan 15 '19 15:01 fkaminski

@fkaminski awesome I will add this to the documentation for this feature!

cytopia avatar Jan 17 '19 08:01 cytopia

Other benefits will include for easier DNS resolution inside the LAN. (e.g. access the Devilbox from various devices: mobiles, tablets,

A little bit confusing: With setup a third network-adapter ( bridged ) in Virtualbox i can reach it within my local LAN by IP ( for example 192.168.178.20 ). (Above is not necessary when i setup Port Forwarding in the VirtualBox)

But this leads me only to the devilbox webinterface.

To use the virtual host there, i had to edit the host file on that computer accordingly. Or, for mobil devices, still use ngrok or xip.io mentioned in #453. Or do i missed something ...

fkaminski avatar Jan 18 '19 08:01 fkaminski

awesome I will add this to the documentation for this feature!

To be not misunderstood. There are two options with Docker Toolbox:

  1. Without Port Forwarding in the VirtualBox:
  • LOCAL_LISTEN_ADDR=
  • HOST_PORT_BIND=53
  • AUTODNS_HOST_ADDRESS=192.168.99.100
  • Set Preferred DNS IP as 192.168.99.100 in the "VirtualBox Host-Only Ethernet Adapter #2"

OR

  1. With Port Forwarding in the VirtualBox (see Screen above):
  • LOCAL_LISTEN_ADDR=
  • HOST_PORT_BIND=53
  • AUTODNS_HOST_ADDRESS=127.0.0.1
  • Set Preferred DNS IP as 127.0.0.1 in the "VirtualBox Host-Only Ethernet Adapter #2"

But how can I reach the virtual hosts in my local LAN with a mobil device now? I read this: https://devilbox.readthedocs.io/en/latest/corporate-usage/shared-devilbox-server-in-lan.html but did not get it ...

On another Laptop within my LAN i can reach the devilbox webinterface by IP. And when i edit the host file on that Laptop manually i can also reach the corresponding virtual hosts.

How can i reach the virtual hosts by hostname with a mobil device without using public services like ngrok (#453)?

Maybe you have some hints on that ... Thanks

fkaminski avatar Jan 18 '19 13:01 fkaminski

How can i reach the virtual hosts by hostname with a mobil device without using public services like ngrok (#453)?

Maybe you have some hints on that ... Thanks

Once AutoDNS feature is merged, it should be as simple as:

  • Change DNS server on your phone (apps for iphone and android should exist) to point to the Devilbox's DNS server on your computer.

cytopia avatar Jan 28 '19 00:01 cytopia

Hm, I'm afraid it's not that easy ... Ich beschreibe es mal auf Deutsch - if you don't mind :-)

In meinem privaten Netzwerk:

Desktop-A:

  • Windows mit Docker-Toolbox
  • Static-IP: 192.168.178.30
  • Docker-IP: 192.168.99.100
  • Bevorzugter DNS-Server: 127.0.0.1

Desktop-B / Mobil-Device:

  • Bevorzugter DNS-Server: 192.168.178.30 ( Desktop-A )

Um Router/Firewall Probleme ausschließen zu können habe ich testweise ( Ohne docker-machine start ) die portable Version von Acrylic DNS Proxy auf Desktop-A gezogen und für *.loc konfiguriert/gestartet.

Im Anschluß funktioniert lokal ( auf Desktop-A ) ping 123.loc/ping 345.loc etc.. Auf Desktop-B funktioniert dann ebenfalls ping 123.loc/ping 345.loc/xyz.loc.

Dann habe ich testweise ( ohne devilbox ) nur docker-bind gestartet mit:

docker run -i -p 53:53/tcp -p 53:53/udp -e WILDCARD_DNS='loc=192.168.178.30' -t cytopia/bind

Auf Desktop-A funktioniert ping 123.loc/ping 345.loc etc.. Leider nicht über Desktop-B?

Nur in Verbindung mit Virtualbox/docker-toolbox ( docker-machine ) bekomme ich keine Auflösung der TLD's im privaten Netzwerk hin ...

Also unabhängig von den devilbox "docker-volumes"/"autodns" branches, komme ich hier nicht weiter... Evtl. kennt jemand dazu Lösungsmöglichkeiten.

fkaminski avatar Jan 28 '19 14:01 fkaminski

I am currently still investigating different approaches. I might as well add DNS functionality directly into the PHP-FPM container, so that it will always resolve the web server in case you e.g.: curl (from inside the container) a project. This would probably be less error prone.

The v1.0.0-alpha1 release will therefore be without the new autodns feature, so that I can start moving forward without being blocked by this one.

cytopia avatar Feb 03 '19 11:02 cytopia

@fkaminski thanks again for the troubleshooting!

I have moved out the AutoDNS feature from v1.0.0 into v1.1.0 and will tackle it in a slightly different approach:

In the first step, I will integrate a DNS server directly into the PHP container so that this will work without external dependencies and point everything automatically to the webserver (or configurable also to varnish or haproxy - depending on the need).

Then the only configuration that needs to be taken care of is the current bind container.

Once that is all tackled and proven stable, I will reconsider moving the PHP container integrated DNS server out for modularity.

cytopia avatar Mar 10 '19 10:03 cytopia