Not using latest blacklists even when completely restarting docker container

[Fooughhy]

Been trying to configure pihole now for a couple of days and it seems to be working ok (though I can't really get the "DeveloperDan's" list to work (but that's another issue).

What my current issue is is that I can't seem to add more lists to the pihole-updateLists container through the environment variables. But maybe I just don't understand how they are supposed to work.

I have this docker-compose.yml:

docker-compose.yml

version: "3"

More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/

services:
    pihole:
        container_name: pihole
        image: jacklul/pihole:latest
        # For DHCP it is recommended to remove these ports and instead add: network_mode: "host"
        #ports:
        #    - "53:53/tcp"
        #    - "53:53/udp"
        #    - "67:67/udp" # Only required if you are using Pi-hole as your DHCP server
        #    - "80:80/tcp"
        network_mode: host
        environment:
            TZ: 'Europe/Stockholm'
            # WEBPASSWORD: '****'
            #ADLISTS_URL: './adlists'
            ADLISTS_URL: 'https://v.firebog.net/hosts/lists.php?type=tick
            https://v.firebog.net/hosts/lists.php?type=nocross
            https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt
            https://small.oisd.nl
            https://big.oisd.nl'
            #WHITELIST_URL: './whitelist'
            WHITELIST_URL: 'https://raw.githubusercontent.com/anudeepND/whitelist/master/domains/whitelist.txt
            https://github.com/anudeepND/whitelist/blob/master/domains/optional-list.txt
            attestation.xboxlive.comcert.mgt.xboxlive.com
            ctldl.windowsupdate.comdef-vef.xboxlive.com
            device.auth.xboxlive.comeds.xboxlive.com
            help.ui.xboxlive.comlicensing.xboxlive.commicrosoft.com
            notify.xboxlive.comsettings-win.data.microsoft.com
            title.auth.xboxlive.comtitle.mgt.xboxlive.com
            v10.vortex-win.data.microsoft.com
            www.msftncsi.com
            xbox.ipv6.microsoft.com
            xboxexperiencesprod.experimentation.xboxlive.com
            xflight.xboxlive.comxkms.xbolive.com
            xsts.auth.xboxlive.com
            v20.events.data.microsoft.com
            watson.telemetry.microsoft.com
            web.vortex.data.microsoft.com
            v10.events.data.microsoft.com
            analytics.svt.se'
            #REGEX_WHITELIST_URL: ''
            #BLACKLIST_URL: './blacklist'
            BLACKLIST_URL: 'https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt'
            REGEX_BLACKLIST_URL: 'https://raw.githubusercontent.com/mmotti/pihole-regex/master/regex.list'
            # Volumes store your data between container upgrades
            GROUP_ID: 1
            PIHOLE_DNS_: '1.1.1.1;1.0.0.1'
        env_file:
            - .env
        volumes:
            - './etc-pihole:/etc/pihole'
            - './etc-dnsmasq.d:/etc/dnsmasq.d'
            # If you need advanced configuration create a mount to access the config file:
            - './etc-pihole-updatelists/:/etc/pihole-updatelists/'
            #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
        cap_add:
            - NET_ADMIN # Required if you are using Pi-hole as your DHCP server, else not needed
        restart: unless-stopped

And when I inspect the running container, I see that the environment contains these lists that I have in the docker-compose.yml file:

docker container inspect pihole

"Env": [
    "BLACKLIST_URL=https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt",
    "REGEX_BLACKLIST_URL=https://raw.githubusercontent.com/mmotti/pihole-regex/master/regex.list",
    "GROUP_ID=1",
    "PIHOLE_DNS_=1.1.1.1;1.0.0.1",
    "TZ=Europe/Stockholm",
    "ADLISTS_URL=https://v.firebog.net/hosts/lists.php?type=tick https://v.firebog.net/hosts/lists.php?type=nocross https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt https://small.oisd.nl https://big.oisd.nl",
    "WEBPASSWORD=Hj41m4rBr4n71ng",
    "WHITELIST_URL=https://raw.githubusercontent.com/anudeepND/whitelist/master/domains/whitelist.txt https://github.com/anudeepND/whitelist/blob/master/domains/optional-list.txt attestation.xboxlive.comcert.mgt.xboxlive.com ctldl.windowsupdate.comdef-vef.xboxlive.com device.auth.xboxlive.comeds.xboxlive.com help.ui.xboxlive.comlicensing.xboxlive.commicrosoft.com notify.xboxlive.comsettings-win.data.microsoft.com title.auth.xboxlive.comtitle.mgt.xboxlive.com v10.vortex-win.data.microsoft.com www.msftncsi.com xbox.ipv6.microsoft.com xboxexperiencesprod.experimentation.xboxlive.com xflight.xboxlive.comxkms.xbolive.com xsts.auth.xboxlive.com v20.events.data.microsoft.com watson.telemetry.microsoft.com web.vortex.data.microsoft.com v10.events.data.microsoft.com analytics.svt.se",
    "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "phpver=php",
    "PHP_ERROR_LOG=/var/log/lighttpd/error-pihole.log",
    "IPv6=True",
    "S6_KEEP_ENV=1",
    "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
    "S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0",
    "FTLCONF_LOCAL_IPV4=0.0.0.0",
    "FTL_CMD=no-daemon",
    "DNSMASQ_USER=pihole"
],

But for some reason the actual lists aren't updated in the DB. If I go to "Whitelists" and e.g. search for svt, it doesn't show up.

Between the edits I make to the docker-compose.yml file, I do docker compose down and then docker compose up -d.

[jacklul]

If you look at your container startup log it will full of errors because your configuration is wrong. That's because ADLISTS_URL is supposed to point to "list of lists" (like this one). Other _URL variables are also wrong, they need to be set to URL that contains individual entries to be inserted (like this one for allowlist).

Your BLACKLIST_URL is also wrong, the URL you provided there is and ADLIST. BLACKLIST_URL is mostly for user-blocked domains and should not be used to load big lists because it will cause major slowdown of the Pi-hole.

Information about how to set those correctly are all in the readme.

[Fooughhy]

Hmm, ok thank you for that quick response. I've used the RPi before, a couple of years ago, and remember having difficulties knowing exactly what defines these different lists then too.

But firstly I guess I need to clear this up. When you say "Adlist" you mean that I have to go into the dashboard of my pihole and enter the lists individually there, right? Instead of using pihole-updateLists in any way really. Because docker-pi-hole doesn't (to my knowledge) have any way of specifying lists in its docker-compose file.

For the IOSD lists, I simply used their guide here: https://oisd.nl/setup/pihole

Their lists are a bit weird, as the links seem to be scripts that download the files directly, instead of allowing me to display them raw in the browser. Do you not think these will work with pihole if I add them as normal ADLISTS in the web interface?

And finally, for the WHITELIST_URL lists, is there no way for pihole-updateLists to use this URL then? https://github.com/anudeepND/whitelist/blob/master/domains/optional-list.txt

[Fooughhy]

And, btw, should the "pihole-updatelists.conf" file in the volume "etc-pihole-updatelists" be empty? Or is it just that when I started the container the first time I had it misconfigured and thus the docker-compose script didn't populate that file, and now that the file exists there will be no further additions to it through the environment variables in the docker-compose file?

[jacklul]

This is the corrected configuration:

        environment:
            ADLISTS_URL: 'https://v.firebog.net/hosts/lists.php?type=tick https://v.firebog.net/hosts/lists.php?type=nocross'
            WHITELIST_URL: 'https://raw.githubusercontent.com/anudeepND/whitelist/master/domains/whitelist.txt https://github.com/anudeepND/whitelist/blob/master/domains/optional-list.txt'
            REGEX_BLACKLIST_URL: 'https://raw.githubusercontent.com/mmotti/pihole-regex/master/regex.list'

(I am unsure if line breaks are valid in that file format so I'm using space as a separator)

...and you add these in the Pi-hole interface directly:

https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt
https://small.oisd.nl
https://big.oisd.nl

You do not place links to individual adlists in ADLISTS_URL. You do use adlists with BLACKLIST_URL because it will slowdown the Pi-hole (those entries will not be in the Pi-hole's gravity b-tree).

When you say "Adlist" you mean that I have to go into the dashboard of my pihole and enter the lists individually there, right?

No, I mean a list like this one.

And, btw, should the "pihole-updatelists.conf" file in the volume "etc-pihole-updatelists" be empty?

Yes. If you're using the environment variables in the docker-compose file they override the config, you don't even have to add the volume - that's why it's commented by default. The option to mount the volume to access the config is for people who need advanced configuration which is not available through the environment variables.

Or is it just that when I started the container the first time I had it misconfigured and thus the docker-compose script didn't populate that file, and now that the file exists there will be no further additions to it through the environment variables in the docker-compose file?

The script first loads the config then parses environment variables which override config ones. The config is not written to.