[FEATURE] Options for using pigz for faster backups in command_backup.sh

[drphlux]

• Feature request copied from "Discussions"

Hello. Pigz compresses using threads to make use of multiple processors and cores.

tar --use-compress-program=pigz
tar cf - paths-to-archive | pigz -9 -p 32 > archive.tar.gz

This could possibly benefit more users, speeding up the backup process. Maybe even enable options for describing how many cores to use? If you still want to use only one cpu core for example.

Here are some examples from my server.

Regular backup using one core with gzip:

vhserver@valheim:~$ ./vhserver backup
[  OK  ] Backing up vhserver: Backup starting
* Previous backup was created less than 1 day ago, total size 813M
[  OK  ] Backing up vhserver: Completed: vhserver-2021-03-04-114952.tar.gz, total size 813M
* Total compression time: 44.19 seconds with 105% CPU usage

Using eight cores with pigz:

vhserver@valheim:~$ ./vhserver backup
[  OK  ] Backing up vhserver: Backup starting
* Previous backup was created less than 1 day ago, total size 813M
[  OK  ] Backing up vhserver: Completed: vhserver-2021-03-04-115622.tar.gz, total size 812M
* Total compression time: 8.86 seconds with 793% CPU usage

Bonus - using all eight cores with zstd (zstdmt):

vhserver@valheim:~$ ./vhserver backup
[  OK  ] Backing up vhserver: Backup starting
* Previous backup was created less than 1 day ago, total size 798M
[  OK  ] Backing up vhserver: Completed: vhserver-2021-03-04-124623.tar.zst, total size 798M
* Total compression time: 2.08 seconds with 406% CPU usage

The actual code:

compression_time=`/usr/bin/time -f "* Total compression time: %e seconds with %P CPU usage" \
                tar -I zstdmt -cf "${backupdir}/${backupname}.tar.zst" \
                -C "${rootdir}" --exclude "${excludedir}" --exclude "${lockdir}/backup.lock" ./. \
                2>&1 > /dev/null`

[whytf]

Very useful feature +1