docker-pi-hole

docker-pi-hole copied to clipboard

Versions

• Pi-hole: v5.1.2
• AdminLTE: v5.1.1
• FTL: V5.2

Platform

• OS and version: Debian 10

• Platform: Docker

Expected behavior

Backup is accepted by teleporter and adlists and other settings are imported.

Actual behavior / bug

No data is imported. Resulting screen shows:

Processed adlist (0 entries) Processed adlist group assignments (0 entries) Processed blacklist (exact) (0 entries) Processed blacklist (regex) (0 entries) Processed client (0 entries) Processed client group assignments (0 entries)

Steps to reproduce

Steps to reproduce the behavior:

1. have Pihole installed natively on a Raspberry Pi 3B (Pihole Versioning match the ones listed above)
2. Navigate to teleporter tab within the settings section of the pihole web console
3. Backup data from Pi using teleporter
4. Navigate to the web console run on the Debian docker container
5. Upload archived backup files and attempt to restore.
6. See page indicating nothing has changed. 4.5. Additionally it can be seen that the adlists are not updated in the group-adlists page

Debug Token

• URL: https://tricorder.pi-hole.net/acblujzjum

Additional context

the backup from the Raspberry Pi 3B was successfully imported by PiHole running on a Raspberry Pi Zero W

I am also seeing this behavior. Here is the lighthttpd error log.

root@fc733ed84a95:/# tail -f /var/log/lighttpd/error.log 
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3Stmt::execute(): Unable to execute statement: unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 202
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3Stmt::execute(): Unable to execute statement: unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 202
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3Stmt::execute(): Unable to execute statement: unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 202
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): no such table: whitelist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3Stmt::execute(): Unable to execute statement: unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 202
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): no such table: regex_whitelist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90

Edit: I took a look at the sqlite tables in the container and the missing tables from the warnings all have vw_. I think the teleporter.php script needs to be updated.

root@a2fcb4cdd587:/# sqlite3 /etc/pihole/gravity.db
SQLite version 3.16.2 2017-01-06 16:32:41
Enter ".help" for usage hints.
sqlite> .tables
adlist               domainlist_by_group  vw_gravity         
adlist_by_group      gravity              vw_regex_blacklist 
client               group                vw_regex_whitelist 
client_by_group      info                 vw_whitelist       
domain_audit         vw_adlist          
domainlist           vw_blacklist

Note to devs, anyone who tries to teleport from an existing instance is wiping their database tables as a result of this. I tried doing this from one instance to another, and now I have to resinstall my pihole

is there a workaround for this?

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

Hi @PromoFaux, may I ask you to properly label this issue to avoid any automation like the stale one?

I think this issue to be still relevant for those using the Teleporter feature as described above. At least, we should temporarily disable the feature when running on a container.

TIA, Matteo

Sure thing. The reason we enabled the stalebot was because of an overwhelming number of open issues (some of which were solved just never closed). Effectively a scream test - if they're closed and someone speaks up, then we probably still need to deal with it 🙃

I

Reading through the above comments. I wonder if this is the same issue as we've seen in other issues where file ownerships weren't set correctly in startup...

Does this still happen on the :Dev tag? (I will also try to repro later)

Hi @PromoFaux; my guess is that:

1. Permissions (RW) should be handled as a potential issue in order to warn the user about the inability to perform the action due to that reason. Here if the container could have a responsibility on this permissions issue and there to manage the permissions issue from PHP POV
2. Database prefix should be handled too. I do see nothing about tables prefix in https://docs.pi-hole.net/database/ftl/, though.

In my own installation - Docker Tag 2022.01.1 - I got a different error:

$ docker exec -it pihole bash -c "tail -f /var/log/lighttpd/error.log"
2022-01-07 09:28:24: (server.c.1464) server started (lighttpd/1.4.53)
2022-02-12 10:33:20: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning:  SQLite3::exec(): no such table: blacklist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2022-02-12 10:33:20: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning:  SQLite3::exec(): no such table: regex_blacklist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2022-02-12 10:33:20: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning:  SQLite3::exec(): no such table: whitelist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2022-02-12 10:33:20: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning:  SQLite3::exec(): no such table: regex_whitelist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90

while reading on the browser:

Processed adlist (14 entries)
Processed adlist group assignments (14 entries)
Processed blacklist (exact) (0 entries)
Processed blacklist (regex) (0 entries)
Processed client (0 entries)
Processed client group assignments (0 entries)
Processed local DNS records (0 entries)
Processed domain_audit (1 entry)
Processed black-/whitelist group assignments (112 entries)
Processed group (2 entries)
Processed whitelist (exact) (110 entries)
Processed whitelist (regex) (2 entries)
OK

Edit @2022-02-12 10:40, and my sqlite DB looks like:

sqlite> .tables
adlist               domainlist_by_group  vw_gravity
adlist_by_group      gravity              vw_regex_blacklist
client               group                vw_regex_whitelist
client_by_group      info                 vw_whitelist
domain_audit         vw_adlist
domainlist           vw_blacklist

(Forgive my ignorance about these tables and the vw table prefix 😉)

HTH, Matteo

So, with a fresh container using 2021.01.01, (and indeed, dev and nightly tags), the only thing I am able to repo is:

2022-02-12 11:03:02: mod_fastcgi.c.487) FastCGI-stderr:PHP Warning:  SQLite3::exec(): no such table: blacklist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2022-02-12 11:03:02: mod_fastcgi.c.487) FastCGI-stderr:PHP Warning:  SQLite3::exec(): no such table: regex_blacklist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2022-02-12 11:03:02: mod_fastcgi.c.487) FastCGI-stderr:PHP Warning:  SQLite3::exec(): no such table: whitelist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2022-02-12 11:03:02: mod_fastcgi.c.487) FastCGI-stderr:PHP Warning:  SQLite3::exec(): no such table: regex_whitelist in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90

the vw_ prefix indicates that these are views (think of them as virtual tables)

~~DL6ER - you're probably closer to the gravity db than I, does it look like we need to change something here to squash these warnings?~~ (Edit: Unping - no longer needed)

Other than that - the entries from the Raspberry Pi export are populated into the docker container....

here is the compose file I am playing with FWIW:

# version: "3.3"

services:
  pihole:
    container_name: pihole
    hostname: pihole
    image: pihole/pihole:nightly
    environment:
      TZ: 'Europe/London'
      PIHOLE_DNS_: '1.1.1.1;1.0.0.1'
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "80:80/tcp"
    volumes:
      - './etc-pihole:/etc/pihole'
      - './etc-dnsmasq.d:/etc/dnsmasq.d'
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN
    restart: always
    dns: 8.8.8.8

TNX @PromoFaux!

the vw_ prefix indicates that these are views (think of them as virtual tables)

My bad, I should be able to identify it as a ref to a view 😳. Thanks a ton for spending time in investigating on this issue and on new, unrelated, warnings. Appreciated! 👍

Matteo

It looks like there are two separate issues being discussed in this thread. To keep things tidy - the no such table warnings are being looked at here : https://github.com/pi-hole/AdminLTE/issues/2117

The OP (@The1CABAL) Mentions that the teleporter backup is not restoring at all. A helpful comment above from @smp4488 ) notes:

2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90
2020-10-04 15:37:21: (mod_fastcgi.c.2543) FastCGI-stderr: PHP Warning:  SQLite3::exec(): unable to open database file in /var/www/html/admin/scripts/pi-hole/php/teleporter.php on line 90

Which suggests permission issues... However things have changed in that regard since this issue was originally reported in Oct 2020. So, I have just spun up a fresh container using the :v5.1.2 tag (which was current at the time this issue was created) created a teleporter file from within it (using one from my most recent install caused issues) - and am still unable to reproduce the initially reported issue.

As it stands, I've been able to import teleporter backups with no issues.

To respond to @muzzah :

anyone who tries to teleport from an existing instance is wiping their database tables as a result of this.

The first thing the teleporter process does is flush the tables before reinserting from the tar file. I'm not sure you're seeing the same thing as the OP, as if you had permission issues with the file system, you wouldn't even see the tables being flushed

And this, then, all leads to the question - as there appear to be multiple issues being discussed here, @bufo333 asks:

is there a workaround for this?

Depends on which bit you are asking about.

Is this still an (open) issue? 🤔

As far as I'm aware, no.. I'm using https://github.com/mattwebbio/orbital-sync to keep my two instances in sync and that uses the teleporter function as though you were doing it manually...

So, if this is no issue anymore, why do you not close it? 😉

Because @The1CABAL might still have an issue.... I will close it, we can open again if needed