nautical-backup
nautical-backup copied to clipboard
Minituff
Containers skipped
I get the following in my log file:
Backup Report - 2024-09-19 03:00:01.122325
2024-09-19 03:00:01.139351 - INFO: Starting backup...
2024-09-19 03:00:01.189656 - INFO: Processing 6 containers...
2024-09-19 03:00:01.236685 - INFO: Stopping nautical-backup...
2024-09-19 03:00:01.423824 - INFO: Shutting down API...
My compose file is:
services:
nautical-backup:
image: minituff/nautical-backup:latest
restart: unless-stopped
container_name: nautical-backup
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /opt/nautical-backup/config:/config
- /opt/:/app/source
- /media/files/Settings/Backups/Docker/Iris:/app/destination
- /opt/nautical-backup/ntfy.sh:/ntfy.sh # Mount ntfy script
environment:
- CRON_SCHEDULE=0 3 * * *
- SKIP_CONTRAINERS=nautical-backup
- TZ=America/New_York
- POST_BACKUP_EXEC=/ntfy.sh
and all that is in my backup directory after running is my log file. The ntfy.sh is just a script pushing the output to ntfy.
Hi,
It seems like Nautical is attempting to backup itself. I have some steps in place to prevent this but it seems like that isn't working.
Could you add the following label to your Nautical config:
"nautical-backup.enable=false"
"nautical-backup.stop-before-backup=false"
If this fixes the problem, then I need to troubleshoot further as to why Nautical didn't skip itself.
Could you also set the console LOG_LEVEL to TRACE and share what you find in the Docker console (not the backup report).
Also, great use of the nfty script. I love that service!
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 2 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | INFO: Connected to database at '/config/nautical-db.json'
nautical-backup | /app/logger.sh: line 46: ((: 1 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 2 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 2 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | /app/logger.sh: line 46: ((: 0 >= : syntax error: operand expected (error token is ">= ")
nautical-backup | INFO: Connected to database at '/config/nautical-db.json'
nautical-backup | DEBUG: Verifying source directory '/app/source'...
nautical-backup | TRACE: Source directory '/app/source' READ access verified
nautical-backup | DEBUG: Verifying Nautical destination directory '/app/destination'...
nautical-backup | TRACE: Destination directory '/app/destination' READ/WRITE access verified
nautical-backup | INFO: Starting backup...
nautical-backup | DEBUG: Verifying destination directory '/app/destination'...
nautical-backup | TRACE: Destination directory '/app/destination' READ/WRITE access verified
nautical-backup | INFO: Processing 6 containers...
nautical-backup | DEBUG: Containers: nautical-backup, diun-diun-1, uptime-kuma, komodo_core_1, komodo_ferretdb_1, komodo_periphery_1
nautical-backup | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'minituff/nautical-backup'.
nautical-backup | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'nautical-backup'.
nautical-backup | DEBUG: Skipping nautical-backup based on label
nautical-backup | DEBUG: diun-diun-1 - Source directory '/app/source/diun-diun-1' does not exist. Skipping
nautical-backup | DEBUG: Container diun-diun-1 was not stopped. No need to start.
nautical-backup | INFO: Stopping uptime-kuma...
nautical-backup | DEBUG: Verifying destination directory '/app/destination'...
nautical-backup | TRACE: Destination directory '/app/destination' READ/WRITE access verified
nautical-backup | ERROR: Destination directory '/app/destination/uptime-kuma' does not exist
nautical-backup | INFO: Backing up uptime-kuma...
nautical-backup | DEBUG: RUNNING: 'rsync -raq /app/source/uptime-kuma/ /app/destination/uptime-kuma/'
nautical-backup | INFO: Starting uptime-kuma...
nautical-backup | INFO: Backup of uptime-kuma complete!
nautical-backup | DEBUG: komodo_core_1 - Source directory '/app/source/komodo_core_1' does not exist. Skipping
nautical-backup | DEBUG: Container komodo_core_1 was not stopped. No need to start.
nautical-backup | DEBUG: komodo_ferretdb_1 - Source directory '/app/source/komodo_ferretdb_1' does not exist. Skipping
nautical-backup | DEBUG: Container komodo_ferretdb_1 was not stopped. No need to start.
nautical-backup | DEBUG: komodo_periphery_1 - Source directory '/app/source/komodo_periphery_1' does not exist. Skipping
nautical-backup | DEBUG: Container komodo_periphery_1 was not stopped. No need to start.
nautical-backup | DEBUG: Running POST_BACKUP_EXEC
nautical-backup | TRACE: Setting environment variable NB_EXEC_COMMAND to /ntfy.sh
nautical-backup | TRACE: Setting environment variable NB_EXEC_ATTACHED_TO_CONTAINER to False
nautical-backup | TRACE: Setting environment variable NB_EXEC_CONTAINER_NAME to None
nautical-backup | TRACE: Setting environment variable NB_EXEC_CONTAINER_ID to None
nautical-backup | TRACE: Setting environment variable NB_EXEC_BEFORE_DURING_OR_AFTER to AFTER
nautical-backup | INFO: Running EXEC command: /ntfy.sh
nautical-backup | WARN: Exec command error: % Total % Received % Xferd Average Speed Time Time Time Current
nautical-backup | Dload Upload Total Spent Left Speed
100 332 100 226 100 106 1458 684 --:--:-- --:--:-- --:--:-- 2155
nautical-backup | DEBUG: Exec command output: {"id":"Xqg24Z07XI3E","time":1726815606,"expires":1726858806,"event":"message","topic":"nautical-backups","message":"Iris nautical-backup failed with status 127:\\n/ntfy.sh: line 8: nautical-backup-command: command not found"}
nautical-backup | DEBUG: Containers completed: uptime-kuma
nautical-backup | DEBUG: Containers skipped: nautical-backup, komodo_periphery_1, komodo_ferretdb_1, diun-diun-1, komodo_core_1
nautical-backup | INFO: Completed in 00m 05s
nautical-backup | INFO: Success. 1 containers backed up! 5 skipped.
Backup Report - 2024-09-20 03:00:01.290184
2024-09-20 03:00:01.302373 - INFO: Starting backup...
2024-09-20 03:00:01.347862 - INFO: Processing 6 containers...
2024-09-20 03:00:01.391589 - INFO: Stopping uptime-kuma...
2024-09-20 03:00:05.725986 - ERROR: Destination directory '/app/destination/uptime-kuma' does not exist
2024-09-20 03:00:05.749564 - INFO: Backing up uptime-kuma...
2024-09-20 03:00:06.247634 - INFO: Starting uptime-kuma...
2024-09-20 03:00:06.566118 - INFO: Backup of uptime-kuma complete!
2024-09-20 03:00:06.593174 - INFO: Running EXEC command: /ntfy.sh
2024-09-20 03:00:06.769319 - WARN: Exec command error: % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 332 100 226 100 106 1458 684 --:--:-- --:--:-- --:--:-- 2155
2024-09-20 03:00:06.791666 - INFO: Completed in 00m 05s
2024-09-20 03:00:06.806711 - INFO: Success. 1 containers backed up! 5 skipped.
2024-09-20 07:42:12.298277 - INFO: Shutting down API...
2024-09-20 07:42:19.267723 - INFO: Starting API on port 8069...
Thanks for sending me the logs. I think I may have misunderstood your issue before.
Based on these logs, I can see that Nautical is not trying to back itself up
nautical-backup | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'minituff/nautical-backup'.
nautical-backup | TRACE: Skipping nautical-backup c1f8057178d6b4fa25b84c8c17b06ee7ac5316c7dfc02fa60a2aeb1adc745e1a because it's image matches 'nautical-backup'.
nautical-backup | DEBUG: Skipping nautical-backup based on label
The last log, "Shutting Down", was this caused by you shutting down the container?
I think these logs are the most relevant to your issue.
nautical-backup | DEBUG: Containers: nautical-backup, diun-diun-1, uptime-kuma, komodo_core_1, komodo_ferretdb_1, komodo_periphery_1
nautical-backup | DEBUG: komodo_core_1 - Source directory '/app/source/komodo_core_1' does not exist. Skipping
nautical-backup | DEBUG: komodo_ferretdb_1 - Source directory '/app/source/komodo_ferretdb_1' does not exist. Skipping
nautical-backup | DEBUG: komodo_periphery_1 - Source directory '/app/source/komodo_periphery_1' does not exist. Skipping
Nautical is looking for a source folder that matches the container name. Since all your containers end in _1, this is likely the reason they are getting skipped.
You have a few options to remedy this (You only need to choose 1).
- Use the
OVERRIDE_SOURCE_DIRenviornment variable on the Nautical config.
# Nautical compose
OVERRIDE_SOURCE_DIR=komodo_core_1:komodo_core,komodo_ferretdb_1:komodo_ferretdb,komodo_periphery_1:komodo_periphery
- Update the
container_nameproperty on each of the containers
Example:
services:
komodo_core:
container_name: komodo_core
- Use the
nautical-backup.override-source-dirlabel on each of your containers
services:
komodo_core:
container_name: komodo_core
labels:
- nautical-backup.override-source-dir=komodo_core
So that was after I put the tags in you suggested to keep it from reading nautical, and I'm unsure why my it's reading a _1 on my container names, that isn't in my directory structure or docker-compose.yml files
It looks like the -1 is coming from the project name of the Docker compose project?
Source; https://stackoverflow.com/questions/33045358/docker-compose-image-named-prefix-s-1-instead-of-s
I haven't encountered this before myself, but you may need to do some tinkering to either remove the -1 or add that to the nautical config so it can be recognized.
@Minituff I just wanted to chime in here and say I am also seeing this issue. I am seeing "-1" appended to all my containers which is resulting in skipped backups. I'll attach the logs at the bottom.
I use Portainer to manage some containers, these are created in stacks. I believe portainer stores data in /data/compose/ and the normal /var/lib/docker/volumes
for example:
root@NAS:/home/machine/nautical-backups# ls /var/lib/docker/volumes
metadata.db teslamate-deploy_teslamate-db
teslamate-deploy_mosquitto-conf teslamate-deploy_teslamate-grafana-data
teslamate-deploy_mosquitto-data
and data/compose
root@NAS:/home/machine/nautical-backups# ls /data/compose/*
/data/compose/10:
openvpn_conf
/data/compose/11:
openvpn-data
/data/compose/13:
ccd_master easyrsa_master
/data/compose/2:
app proxy
/data/compose/3:
config db
/data/compose/34:
import
/data/compose/35:
import
/data/compose/36:
import
/data/compose/37:
import
/data/compose/4:
config db
docker-compose:
services:
nautical-backup:
image: minituff/nautical-backup:latest
container_name: nautical-backup
volumes:
- /var/run/docker.sock:/var/run/docker.sock # Required for Docker interaction
- /var/lib/docker/volumes:/app/source:ro # Mount all Docker volumes as the source
- /home/machine/nautical-backups/config:/config # Configuration directory
- /home/machine/nautical-backups/destination:/app/destination # Backup destination
environment:
- TZ=America/New_York
- BACKUP_ON_START=true
- RUN_ONCE=true
- CRON_SCHEDULE_ENABLED=false
- LOG_LEVEL=TRACE
nautical container runtime logs:
DEBUG: REPORT_FILE: true
INFO: Nautical Backup Version: 2.10.4
DEBUG: Built for the platform: linux/amd64
DEBUG: Perparing enviornment variables...
DEBUG: Found defaults.env
DEBUG: CRON_SCHEDULE: 0 4 * * *
DEBUG: USE_DEST_DATE_FOLDER: false
DEBUG: DEST_DATE_PATH_FORMAT: date/container
DEBUG: DEST_DATE_FORMAT: %Y-%m-%d
DEBUG: USE_CONTAINER_BACKUP_DATE: true
DEBUG: USE_DEFAULT_RSYNC_ARGS: true
DEBUG: REQUIRE_LABEL: false
DEBUG: LABEL_PREFIX: nautical-backup
DEBUG: REPORT_FILE_LOG_LEVEL: INFO
DEBUG: REPORT_FILE_ON_BACKUP_ONLY: true
DEBUG: KEEP_SRC_DIR_NAME: true
DEBUG: EXIT_AFTER_INIT: false
DEBUG: LOG_RSYNC_COMMANDS: false
DEBUG: SOURCE_LOCATION: /app/source
DEBUG: DEST_LOCATION: /app/destination
DEBUG: TEST_MODE: -1
DEBUG: HTTP_REST_API_ENABLED: true
DEBUG: HTTP_REST_API_USERNAME: admin
DEBUG: HTTP_REST_API_PASSWORD: password
DEBUG: ADDITIONAL_FOLDERS_WHEN: before
DEBUG: ADDITIONAL_FOLDERS_USE_DEST_DATE_FOLDER: false
DEBUG: NAUTICAL_DB_PATH: /config
DEBUG: NAUTICAL_DB_NAME: nautical-db.json
TRACE: PRE_BACKUP_CURL:
TRACE: POST_BACKUP_CURL:
TRACE: RSYNC_CUSTOM_ARGS:
TRACE: OVERRIDE_SOURCE_DIR:
TRACE: OVERRIDE_DEST_DIR:
TRACE: SELF_CONTAINER_ID:
TRACE: SKIP_CONTAINERS:
TRACE: SKIP_STOPPING:
TRACE: ADDITIONAL_FOLDERS:
DEBUG: SELF_CONTAINER_ID: a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a
INFO: Skipping CRON installation since CRON_SCHEDULE_ENABLED=false
DEBUG: Verifying source directory '/app/source'...
DEBUG: Verifying destination directory '/app/destination'...
INFO: Connected to database at '/config/nautical-db.json'
DEBUG: Installing nautical backup script...
TRACE: ln -s /app/backup.sh /usr/local/bin/nautical
TRACE: TEST_MODE: -1
INFO: Starting backup since BACKUP_ON_START is true
INFO: API listening on port 8069...
INFO: Note - BACKUP_ON_START logs are not available until all containers are processed, however the report file updates in real-time.
INFO: Initialization complete. Awaiting CRON schedule: 0 4 * * *
TRACE: Running: exec with-contenv python3 -m uvicorn app.api.main:app --host 0.0.0.0 --port 8069 --lifespan on --use-colors --log-level 'warning'
INFO: Connected to database at '/config/nautical-db.json'
DEBUG: Verifying source directory '/app/source'...
TRACE: Source directory '/app/source' READ access verified
DEBUG: Verifying Nautical destination directory '/app/destination'...
TRACE: Destination directory '/app/destination' READ/WRITE access verified
INFO: Starting backup...
DEBUG: Verifying destination directory '/app/destination'...
TRACE: Destination directory '/app/destination' READ/WRITE access verified
INFO: Processing 12 containers...
DEBUG: Containers: nautical-backup, zoneminder, zm_db, teslamate-deploy-teslamate-1, ovpn-admin-ovpn-admin-1, ovpn-admin-openvpn-1, portainer_portainer_1, nginx, teslamate-deploy-mosquitto-1, teslamate-deploy-database-1, teslamate-deploy-grafana-1, buildx_buildkit_default
TRACE: Skipping nautical-backup a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a because it's image matches 'minituff/nautical-backup'.
TRACE: Skipping nautical-backup a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a because it's image matches 'nautical-backup'.
TRACE: Skipping nautical-backup a53217a7d28bca65d0c8ad21151863ab03d8d4ed90052c1d1385b3887c420d4a because it's ID is the same as Nautical
DEBUG: zoneminder - Source directory '/app/source/zoneminder' does not exist. Skipping
DEBUG: Container zoneminder was not stopped. No need to start.
DEBUG: zm_db - Source directory '/app/source/zm_db' does not exist. Skipping
DEBUG: Container zm_db was not stopped. No need to start.
DEBUG: teslamate-deploy-teslamate-1 - Source directory '/app/source/teslamate-deploy-teslamate-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-teslamate-1 was not stopped. No need to start.
DEBUG: ovpn-admin-ovpn-admin-1 - Source directory '/app/source/ovpn-admin-ovpn-admin-1' does not exist. Skipping
DEBUG: Container ovpn-admin-ovpn-admin-1 was not stopped. No need to start.
DEBUG: ovpn-admin-openvpn-1 - Source directory '/app/source/ovpn-admin-openvpn-1' does not exist. Skipping
DEBUG: Container ovpn-admin-openvpn-1 was not stopped. No need to start.
DEBUG: portainer_portainer_1 - Source directory '/app/source/portainer_portainer_1' does not exist. Skipping
DEBUG: Container portainer_portainer_1 was not stopped. No need to start.
DEBUG: nginx - Source directory '/app/source/nginx' does not exist. Skipping
DEBUG: Container nginx was not stopped. No need to start.
DEBUG: teslamate-deploy-mosquitto-1 - Source directory '/app/source/teslamate-deploy-mosquitto-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-mosquitto-1 was not stopped. No need to start.
DEBUG: teslamate-deploy-database-1 - Source directory '/app/source/teslamate-deploy-database-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-database-1 was not stopped. No need to start.
DEBUG: teslamate-deploy-grafana-1 - Source directory '/app/source/teslamate-deploy-grafana-1' does not exist. Skipping
DEBUG: Container teslamate-deploy-grafana-1 was not stopped. No need to start.
DEBUG: buildx_buildkit_default - Source directory '/app/source/buildx_buildkit_default' does not exist. Skipping
DEBUG: Container buildx_buildkit_default was not stopped. No need to start.
DEBUG: Containers completed:
DEBUG: Containers skipped: ovpn-admin-openvpn-1, nginx, teslamate-deploy-teslamate-1, zm_db, buildx_buildkit_default, zoneminder, teslamate-deploy-mosquitto-1, ovpn-admin-ovpn-admin-1, portainer_portainer_1, nautical-backup, teslamate-deploy-database-1, teslamate-deploy-grafana-1
INFO: Completed in 00m 00s
INFO: Success. 0 containers backed up! 12 skipped.
INFO: RUN_ONCE is true. Exiting...
nautical config
root@NAS:/home/machine/nautical-backups# cat config/nautical-db.json
{
"created_at": "2024-12-23 02:02:02",
"backup_running": false,
"containers_skipped": 12,
"containers_completed": 0,
"number_of_containers": 12,
"errors": 0,
"last_backup_seconds_taken": 0,
"last_cron": "12/23/24 02:46",
"completed": "0"
}
Hi @IanWardell,
Are you able to share some of your container configs as well?
I'm wondering if you need to apply the container_name property to each of your containers.
Hi @IanWardell,
Are you able to share some of your container configs as well?
I'm wondering if you need to apply the
container_nameproperty to each of your containers.
I haven't added the container_naame property, but I can update them and see what happens.
I did try the OVERRIDE_SOURCE_DIR= method and it didn't work.
services:
nautical-backup:
image: minituff/nautical-backup:latest
container_name: nautical-backup
volumes:
- /var/run/docker.sock:/var/run/docker.sock # Required for Docker interaction
- /data/compose/:/app/source:ro # Mount all Docker volumes as the source
- /home/machine/nautical-backups/config:/config # Configuration directory
- /home/machine/nautical-backups/destination:/app/destination # Backup destination
environment:
- TZ=America/New_York
- BACKUP_ON_START=true
- RUN_ONCE=true
- CRON_SCHEDULE_ENABLED=false
- LOG_LEVEL=TRACE
- OVERRIDE_SOURCE_DIR=teslamate-deploy-teslamate-1:teslamate-deploy-teslamate
#DEBUG: Containers skipped: ovpn-admin-openvpn-1, nginx, teslamate-deploy-teslamate-1, zm_db, buildx_buildkit_default, zoneminder, teslamate-deploy-mosquitto-1, ovpn-admin-ovpn-admin-1, portainer_portainer_1, nautical-backup, teslamate-deploy-database-1, teslamate-deploy-grafana-1
here is an example docker-compose
version: "3"
services:
teslamate:
image: teslamate/teslamate:latest
restart: always
environment:
- ENCRYPTION_KEY=[REDACTED]
- DATABASE_USER=[REDACTED]
- DATABASE_PASS=[REDACTED]
- DATABASE_NAME=teslamate
- DATABASE_HOST=database
- MQTT_HOST=mosquitto
ports:
- "127.0.0.1:4001:4000"
volumes:
- ./import:/opt/app/import
cap_drop:
- all
database:
image: postgres:15
restart: always
environment:
- POSTGRES_USER=[REDACTED]
- POSTGRES_PASSWORD=[REDACTED]
- POSTGRES_DB=teslamate
volumes:
- teslamate-db:/var/lib/postgresql/data
grafana:
image: teslamate/grafana:latest
restart: always
environment:
- DATABASE_USER=[REDACTED]
- DATABASE_PASS=[REDACTED]
- DATABASE_NAME=teslamate
- DATABASE_HOST=database
ports:
- 3000:3000
volumes:
- teslamate-grafana-data:/var/lib/grafana
mosquitto:
image: eclipse-mosquitto:2
restart: always
command: mosquitto -c /mosquitto-no-auth.conf
# ports:
# - 1883:1883
volumes:
- mosquitto-conf:/mosquitto/config
- mosquitto-data:/mosquitto/data
volumes:
teslamate-db:
teslamate-grafana-data:
mosquitto-conf:
mosquitto-data:
@Minituff
Can you verify what /source needs to be? And should I be listing a source for each docker-compose stack and container?
I have volumes in a few places ex:
- /var/lib/docker/volumes
- /data/compose/* [ ex 33/* or 3/* where each portainer stack puts the volumes and compose files]
rename done:
machine@NAS:~/NAS-Dockerfiles/portainer$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
84c85ee747ab portainer/portainer-ce "/portainer -H unix:…" 40 seconds ago Up 38 seconds 8000/tcp, 9443/tcp, 0.0.0.0:9000->9000/tcp portainer_portainer
e479126d93f8 flant/ovpn-admin:latest "/app/ovpn-admin" 3 minutes ago Up 3 minutes ovpn-admin-ovpn-admin
6ae802cff2c4 openvpn:local "/etc/openvpn/setup/…" 3 minutes ago Up 3 minutes 0.0.0.0:7777->1194/tcp, 127.0.0.1:8888->8080/tcp ovpn-admin-openvpn
ef8f2eebce75 postgres:15 "docker-entrypoint.s…" 9 minutes ago Up 8 minutes 5432/tcp teslamate-deploy-database
edb8a5d92143 teslamate/teslamate:latest "tini -- /bin/sh /en…" 9 minutes ago Up 8 minutes 127.0.0.1:4001->4000/tcp teslamate-deploy-teslamate
8b413e1be965 teslamate/grafana:latest "/run.sh" 9 minutes ago Up 8 minutes 0.0.0.0:3000->3000/tcp teslamate-deploy-grafana
c837d918ab1e eclipse-mosquitto:2 "/docker-entrypoint.…" 9 minutes ago Up 8 minutes 1883/tcp teslamate-deploy-mosquitto
19ff8942c4cf quantumobject/docker-zoneminder:latest "/sbin/my_init" 13 hours ago Up 12 hours 6802/tcp, 9000/tcp, 0.0.0.0:9080->80/tcp zoneminder
87575c8a298a mysql/mysql-server:5.7 "/entrypoint.sh mysq…" 13 hours ago Up 12 hours (healthy) 3306/tcp, 33060/tcp zm_db
020f9bec71b6 nginx:latest "/docker-entrypoint.…" 11 months ago Up 12 hours 0.0.0.0:4000->4000/tcp, 80/tcp, 0.0.0.0:8080-8081->8080-8081/tcp nginx
5b5eeb9cffd9 moby/buildkit:buildx-stable-1 "buildkitd" 17 months ago Up 12 hours buildx_buildkit_default
debug output:
Lines
Actions
DEBUG: REPORT_FILE: true
INFO: Nautical Backup Version: 2.10.4
DEBUG: Built for the platform: linux/amd64
DEBUG: Perparing enviornment variables...
DEBUG: Found defaults.env
DEBUG: CRON_SCHEDULE: 0 4 * * *
DEBUG: USE_DEST_DATE_FOLDER: false
DEBUG: DEST_DATE_PATH_FORMAT: date/container
DEBUG: DEST_DATE_FORMAT: %Y-%m-%d
DEBUG: USE_CONTAINER_BACKUP_DATE: true
DEBUG: USE_DEFAULT_RSYNC_ARGS: true
DEBUG: REQUIRE_LABEL: false
DEBUG: LABEL_PREFIX: nautical-backup
DEBUG: REPORT_FILE_LOG_LEVEL: INFO
DEBUG: REPORT_FILE_ON_BACKUP_ONLY: true
DEBUG: KEEP_SRC_DIR_NAME: true
DEBUG: EXIT_AFTER_INIT: false
DEBUG: LOG_RSYNC_COMMANDS: false
DEBUG: SOURCE_LOCATION: /app/source
DEBUG: DEST_LOCATION: /app/destination
DEBUG: TEST_MODE: -1
DEBUG: HTTP_REST_API_ENABLED: true
DEBUG: HTTP_REST_API_USERNAME: admin
DEBUG: HTTP_REST_API_PASSWORD: password
DEBUG: ADDITIONAL_FOLDERS_WHEN: before
DEBUG: ADDITIONAL_FOLDERS_USE_DEST_DATE_FOLDER: false
DEBUG: NAUTICAL_DB_PATH: /config
DEBUG: NAUTICAL_DB_NAME: nautical-db.json
TRACE: PRE_BACKUP_CURL:
TRACE: POST_BACKUP_CURL:
TRACE: RSYNC_CUSTOM_ARGS:
TRACE: OVERRIDE_SOURCE_DIR:
TRACE: OVERRIDE_DEST_DIR:
TRACE: SELF_CONTAINER_ID:
TRACE: SKIP_CONTAINERS:
TRACE: SKIP_STOPPING:
TRACE: ADDITIONAL_FOLDERS:
DEBUG: SELF_CONTAINER_ID: 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8
INFO: Skipping CRON installation since CRON_SCHEDULE_ENABLED=false
DEBUG: Verifying source directory '/app/source'...
DEBUG: Verifying destination directory '/app/destination'...
INFO: Connected to database at '/config/nautical-db.json'
DEBUG: Installing nautical backup script...
TRACE: ln -s /app/backup.sh /usr/local/bin/nautical
TRACE: TEST_MODE: -1
INFO: API listening on port 8069...
INFO: Starting backup since BACKUP_ON_START is true
INFO: Initialization complete. Awaiting CRON schedule: 0 4 * * *
TRACE: Running: exec with-contenv python3 -m uvicorn app.api.main:app --host 0.0.0.0 --port 8069 --lifespan on --use-colors --log-level 'warning'
INFO: Note - BACKUP_ON_START logs are not available until all containers are processed, however the report file updates in real-time.
INFO: Connected to database at '/config/nautical-db.json'
DEBUG: Verifying source directory '/app/source'...
TRACE: Source directory '/app/source' READ access verified
DEBUG: Verifying Nautical destination directory '/app/destination'...
TRACE: Destination directory '/app/destination' READ/WRITE access verified
INFO: Starting backup...
DEBUG: Verifying destination directory '/app/destination'...
TRACE: Destination directory '/app/destination' READ/WRITE access verified
INFO: Processing 12 containers...
DEBUG: Containers: nautical-backup, zoneminder, zm_db, portainer_portainer, ovpn-admin-ovpn-admin, ovpn-admin-openvpn, teslamate-deploy-database, teslamate-deploy-teslamate, teslamate-deploy-grafana, teslamate-deploy-mosquitto, nginx, buildx_buildkit_default
TRACE: Skipping nautical-backup 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8 because it's image matches 'minituff/nautical-backup'.
TRACE: Skipping nautical-backup 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8 because it's image matches 'nautical-backup'.
TRACE: Skipping nautical-backup 711e4aa9b9319a7f2a4b791c3becbb40bc2a70861e7f6ced67142823a1cfa2c8 because it's ID is the same as Nautical
DEBUG: zoneminder - Source directory '/app/source/zoneminder' does not exist. Skipping
DEBUG: Container zoneminder was not stopped. No need to start.
DEBUG: zm_db - Source directory '/app/source/zm_db' does not exist. Skipping
DEBUG: Container zm_db was not stopped. No need to start.
DEBUG: portainer_portainer - Source directory '/app/source/portainer_portainer' does not exist. Skipping
DEBUG: Container portainer_portainer was not stopped. No need to start.
DEBUG: ovpn-admin-ovpn-admin - Source directory '/app/source/ovpn-admin-ovpn-admin' does not exist. Skipping
DEBUG: Container ovpn-admin-ovpn-admin was not stopped. No need to start.
DEBUG: ovpn-admin-openvpn - Source directory '/app/source/ovpn-admin-openvpn' does not exist. Skipping
DEBUG: Container ovpn-admin-openvpn was not stopped. No need to start.
DEBUG: teslamate-deploy-database - Source directory '/app/source/teslamate-deploy-database' does not exist. Skipping
DEBUG: Container teslamate-deploy-database was not stopped. No need to start.
DEBUG: teslamate-deploy-teslamate - Source directory '/app/source/teslamate-deploy-teslamate' does not exist. Skipping
DEBUG: Container teslamate-deploy-teslamate was not stopped. No need to start.
DEBUG: teslamate-deploy-grafana - Source directory '/app/source/teslamate-deploy-grafana' does not exist. Skipping
DEBUG: Container teslamate-deploy-grafana was not stopped. No need to start.
DEBUG: teslamate-deploy-mosquitto - Source directory '/app/source/teslamate-deploy-mosquitto' does not exist. Skipping
DEBUG: Container teslamate-deploy-mosquitto was not stopped. No need to start.
DEBUG: nginx - Source directory '/app/source/nginx' does not exist. Skipping
DEBUG: Container nginx was not stopped. No need to start.
DEBUG: buildx_buildkit_default - Source directory '/app/source/buildx_buildkit_default' does not exist. Skipping
DEBUG: Container buildx_buildkit_default was not stopped. No need to start.
DEBUG: Containers completed:
DEBUG: Containers skipped: teslamate-deploy-database, zm_db, ovpn-admin-ovpn-admin, ovpn-admin-openvpn, zoneminder, teslamate-deploy-mosquitto, nginx, nautical-backup, teslamate-deploy-teslamate, buildx_buildkit_default, portainer_portainer, teslamate-deploy-grafana
INFO: Completed in 00m 00s
INFO: Success. 0 containers backed up! 12 skipped.
INFO: RUN_ONCE is true. Exiting...
Yeah it would be helpful if you could list out a few of your compose files. I am mostly looking to see how you mount your volumes to each of your apps.
Currently, Nautical doesnt really support native docker files--it's really been designed for bind mounts. So Nautical cant see the volumes in /var/lib/docker/volumes yet.
So lets take a look at 2 of your configs and how we can update them.
database:
image: postgres:15
container_name: postgresql. # Added Name
restart: always
environment:
- POSTGRES_USER=[REDACTED]
- POSTGRES_PASSWORD=[REDACTED]
- POSTGRES_DB=teslamate
volumes:
# - teslamate-db:/var/lib/postgresql/data #OLD
- /home/machine/docker_volumes/postgresql:var/lib/postgresql/data #NEW
grafana:
image: teslamate/grafana:latest
container_name: grafane # Added name
restart: always
environment:
- DATABASE_USER=[REDACTED]
- DATABASE_PASS=[REDACTED]
- DATABASE_NAME=teslamate
- DATABASE_HOST=database
ports:
- 3000:3000
volumes:
# - teslamate-grafana-data:/var/lib/grafana # OLD
- /home/machine/docker_volumes/grafana:/var/lib/grafana # NEW
services:
nautical-backup:
image: minituff/nautical-backup:latest
container_name: nautical-backup
volumes:
- /var/run/docker.sock:/var/run/docker.sock # Required for Docker interaction
# - /data/compose/:/app/source:ro # Mount all Docker volumes as the source (OLD)
- /home/machine/docker_volumes/app/source:ro # Mount all Docker volumes as the source (NEW)
- /home/machine/nautical-backups/config:/config # Configuration directory
- /home/machine/nautical-backups/destination:/app/destination # Backup destination
environment:
# Anything here
The goal is that the /app/source folder inside Nautical will look like this:
/app/souce:
- grafana/
- postgresql/
- other containers .....
These paths would match the container names so nautical knows how to associate the containers.
I am working on Nautical version 3, which would actually read the Compose files and even look inside the /var/lib* folders for the container volumes, but right now that's not working quite yet.
I'm managing my stacks with Dockge. And I'm also seeing odd behavior with container names. For example, I have some named like "stirling-pdf-stirling-pdf-1" and "cloudflared-cloudflared-1" and "pocket-id-pocket-id-1". But not all are doing this. Almost seems like the older ones are so I wonder if it was a Dockge thing.
***Duh. The normal names have an explicit "container_name" setting in the compose file. That's all it was.
But it would be nice if Nautical could deal with these easily. What's the best solution at this point? For example, when using Dockge, the config and data reside in a folder "/opt/stacks/karakeep". The container names for this stack are:
karakeep-web-1 karakeep-chrome-1 karakeep-meilisearch-1
I figured this out using the Immich example on the docs wiki. But in my case of using Dockge, and wanting to backup a folder and it's contents that use the name of the stack, so I thought it might be better to use the group function and just one line on one service for the folder, like this example: nautical-backup.additional-folders=immich. But it doesn't seem to be backing the folder up.