rsync working as intended

[wedge-kc]

Thanks for the software. Very handy.

I am having an issue with rsync. According the log, my custom rsync label in Emby is being used to backup the volume.

2024-04-30 04:00:04.524252 - INFO: Backing up emby...
2024-04-30 04:00:04.524450 - DEBUG: RUNNING: 'rsync -raq --exclude='cache' --exclude='transcoding-temp' --exclude='.log' --exclude='.txt' /app/source/emby/ /app/destination/2024-04-30/emby/'
2024-04-30 04:00:05.953838 - INFO: Starting emby...
2024-04-30 04:00:06.283771 - INFO: Backup of emby complete!

However, the excluded directories, *.log and *.txt are all still copied to the destination directory as if the custom command was not used. If I use the same rsync command from the log and run it inside the container, it seems to work correctly and excludes the desired directories and files. Thanks.

services:
  nautical-backup:
    image: minituff/nautical-backup:2.3
    container_name: nautical-backup
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /home/chris/docker:/app/source
      - /mnt/media/nautical-backup:/app/destination
    environment: # Optional variables 
      - TZ=Asia/Phnom_Penh
      - CRON_SCHEDULE=0 4 * * *
      - USE_DEST_DATE_FOLDER=true
      - REPORT_FILE_LOG_LEVEL=DEBUG
      - POST_BACKUP_CURL=tar -czf /app/destination/$(date +"%Y-%m-%d").tar.gz /app/destination/$(date +"%Y-%m-%d") | xargs rm -R /app/destination/$(date +"%Y-%m-%d")
      - SKIP_CONTAINERS=nautical-backup,portainer

[Minituff]

Interesting. So you said also also exec-ed into Nautical and ran the command too? And this worked?

docker exec -it nautical-backup
rsync -raq --exclude='cache' --exclude='transcoding-temp' --exclude='.log' --exclude='.txt' /app/source/emby/ /app/destination/2024-04-30/emby/

If that's the case I'm thinking it maybe be related to how Python actually calls the rsync command with subprocess.run. Here is the related code:

def _run_rsync(self, c: Optional[Container], rsync_args: str, src_dir: Path, dest_dir: Path):
    src_folder = f"{src_dir.absolute()}/"
    dest_folder = f"{dest_dir.absolute()}/"

    command = f"{rsync_args} {src_folder} {dest_folder}"

    self.log_this(f"RUNNING: 'rsync {command}'", "DEBUG")

    args = command.split()  # Split the command into a list of arguments

    out = subprocess.run(args, shell=True, executable="/usr/bin/rsync", capture_output=False)

Really clever use of the POST_BACKUP_CURL. Ill probably make like a POST_BACKUP_COMMAND so you can do both :).

Also, did you have to place Nautical in the SKIP_CONTAINERS list because it was stopping itself, or was this just precautionary? If Nautical was stopping itself then I need to fix that too.

[wedge-kc]

Interesting. So you said also also exec-ed into Nautical and ran the command too? And this worked?

Correct.

If that's the case I'm thinking it maybe be related to how Python actually calls the rsync command with subprocess.run. Here is the related code:

Thanks for the hint. After messing around a bit more, I was able to get rsync to exclude the directories I wanted by adding the label like this:

nautical-backup.rsync-custom-args=--exclude=cache --exclude=transcoding-temp --exclude=*.txt

Really clever use of the POST_BACKUP_CURL. Ill probably make like a POST_BACKUP_COMMAND so you can do both :).

Thanks. Both a POST and PRE command would be useful. I was going to add a PRE_BACKUP_CURL command to also delete older backups to make everything automatic.

Also, did you have to place Nautical in the SKIP_CONTAINERS list because it was stopping itself, or was this just precautionary? If Nautical was stopping itself then I need to fix that too.

Yes. Nautical shut itself down if I didn't add it to the SKIP_CONTAINERS list.

Thanks for the help.

[Minituff]

I was able to get rsync to exclude the directories I wanted

Awesome, glad it's working for you! Since you also exec-ed into Nautical and ran the rsync command manually--I do want that to work inside the backup script as well, so I might do some more testing on this.

Yes. Nautical shut itself down if I didn't add it to the SKIP_CONTAINERS list.

Great catch!! This issue should be fixed in the latest release.