[Question] How to pihole+dhcp+DoH over docker swarm

[bluefangs]

Hello,

I'm a novice to docker/swarm and portainer. I've managed to run portainer in a standalone mode an have been successful at installing pihole + cloudflared DoH in standalone docker-compose. So, I began to delve into the world of swarms and am looking to see if I can move the above setup into a swarm - something that is herculean to me. After wandering the web looking for a solution for a week, I stumbled upon your version of installing pi + cloudflared over a docker swarm.

Existing setup for pihole-dhcp-dns-doh:

version: "2"
services:
    pihole:
        container_name: pihole
        image: pihole/pihole:latest
        restart: unless-stopped
        ports:
            - "53:53/tcp"
            - "53:53/udp"
            - "67:67/udp"
            - "80:80/tcp"
            - "443:443/tcp"
        environment:
            - ServerIP=192.168.0.100
            - TZ='Asia/Kolkata'
            - DNS1='10.0.0.2#5054'
            - DNS2=''
            - IPv6=false
            - DNSMASQ_LISTENING=all
            - WEBPASSWORD=some_password   #................................ set a secure password here or it will be random
        # Volumes store your data between container upgrades
        volumes:
        - '/usr/local/etc/pihole/pihole-data/:/etc/pihole/'
        - '/usr/local/etc/pihole/pihole-data/dnsmasq.d/:/etc/dnsmasq.d/'
        dns:
            - 127.0.0.1
            - 1.1.1.1
        # Recommended but not required (DHCP needs NET_ADMIN)
        #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
        cap_add:
            - NET_ADMIN
        network_mode: host

    cloudflared:
        container_name: cloudflared
        # image: fardog/secureoperator:latest # google dns
        image: visibilityspots/cloudflared:latest # cloudflare dns
        restart: unless-stopped
        dns:
            - 1.1.1.1
            - 1.0.0.1
        networks:
            pihole_net:
                ipv4_address: 10.0.0.2
networks:
    pihole_net:
        driver: bridge
        ipam:
            config:
                - subnet: 10.0.0.0/29   #..................................... /29 can accomodate 6 hosts

I brought down the above setup, set up portainer as a swarm as per the original documentation and then tried to add the below (based off your implementation - albeit using a different cloudflared image (visibilityspots/cloudflared).

Swarm config:

version: "3.7"
services:
  pihole:
    image: pihole/pihole:latest
    hostname: "pihole"    
    deploy:
      replicas: 1
      placement: 
          constraints: [node.role == manager]
      update_config:
        delay: 30s
    # For DHCP it is recommended to remove these ports and instead add: network_mode: "host"
    ports:		# <<<<<< as per the above comments, if I remove the ports, I'm unable to access pihole web interface
      - target: 53
        published: 53
        protocol: tcp
      - target: 53
        published: 53
        protocol: udp
      - target: 67
        published: 67
        protocol: udp
      - target: 80
        published: 80
        protocol: tcp
      - target: 443
        published: 443
        protocol: tcp  
    network_mode: "host" # <<<<<< as per the above comments, I've enabled 'host' network_mode.
    environment:
      TZ: "Asia/Kolkata"
      WEBPASSWORD: "some_password"
      DNS1: "172.23.0.1#5053" # <<<<<< This is my docker_gwbridge gateway IP
      # DNS2: "172.18.0.1#5053"
      DNS_FQDN_REQUIRED: "true"
      DNS_BOGUS_PRIV: "true"
      DNSSEC: "false"
      # CONDITIONAL_FORWARDING: "true"	# <<<<<< not sure if this and the below are reqd as I've disabled DHCP my the router
      # CONDITIONAL_FORWARDING_IP: "192.168.0.1"
      # CONDITIONAL_FORWARDING_DOMAIN: "home.local"
      # CONDITIONAL_FORWARDING_REVERSE: "0.168.192.in-addr.arpa"
    # Volumes store your data between container upgrades
    volumes:
      - /usr/local/etc/pihole/pihole-data:/etc/pihole/
      - /usr/local/etc/pihole/pihole-data/dnsmasq.d:/etc/dnsmasq.d/
    dns:
      - "127.0.0.1"
      #- "1.1.1.1" # fallback

  cloudflared:
    # Pinned to arm/v7 digest for crazymax/cloudflared:2020.2.0
    image: visibilityspots/cloudflared:latest # <<<<<< using this image instead of the one you've suggested
    deploy:
      replicas: 1
      placement: 
          constraints: [node.role == manager]
      update_config:
        delay: 30s    
    ports:
      - target: 5053
        published: 5053
        protocol: udp
      - target: 49312	# <<<<<< what is this for?
        published: 49312
        protocol: tcp
    environment:
      - "TZ=Asia/Kolkata"
      - "TUNNEL_DNS_UPSTREAM=https://1.1.1.1/dns-query,https://1.0.0.1/dns-query"

I"ve mentioned some comments in the above swarm-config as per my level of understanding / edits. I was able to start the above via portainer, however, I ran into the multiple issues.

1. DHCP server was not doing it's job as I lost internet access
2. Pi was still able to access the internet, however, resolution was not happening via 1.1.1.1/DoH as per https://1.1.1.1/help

Do you happen to see any faulty configurations?

I was inspired to set up something similar to this blog wherein he talks about DNS server redundancy.

Thanks