helloSystem
Backup utility
We need a Backup utility that can make backups to local media, the local area network, and to Cloud providers. Probably the initial focus should be on Cloud Backup.
Cloud Backup
Requirements
- All data including all metadata needs to be encrypted so that the cloud provider cannot peek into the data
- GUI should offer an option to shut down the machine after the upload
Possible underlying tools
- zfs send/receive (native to zfs but also dependent on it, hence not really cross-platform; mature?)
- restic (optimized for speed, Golang, cross-platform, not zfs dependent; not 1.0 yet. How risky is purging data that is no longer needed on the server?)
Possible cloud service providers
- Tarsnap
- https://www.backblaze.com/ (seems affordable, ~1 EUR per month for a normal workstation)
Under a broader heading e.g. Backup options …
From https://github.com/helloSystem/hello/issues/32#issue-730983594:
…
ZFS features I want to make use of
- …
- …
- Replication (network backups): I want to backup (image) the whole system to a network device, which I can restore from ("bare metal") like with a Time Machine network backup. Doing subsequent backups should copy only what has changed ("delta backups"). PC-BSD had a graphical installer that could restore systems from network servers (e.g., FreeNAS) using ZFS replication over SSH. I would want to re-create something like that.
Ideally I want to configure as little as possible as I am not a system administrator but a mere end user (zero configuration). …
sysutils/life-preserver
Port deleted around a year ago – abandoned upstream (https://github.com/trueos/lpreserver archived) – but let's toy with the possibility of forking and resurrecting this, with the aim of:
- someone making it as close as reasonably possible to zero configuration.
The GUI was relatively easy to use in TrueOS Desktop. From TrueOS 12: a review – Developers Log (2017-04-18):
From what I recall of earlier use of the OS, it was relatively easy to configure the security that was appropriate/required for network backups. https://www.phoronix.com/forums/forum/software/bsd-mac-os-x-hurd-others/896297-trying-out-the-freebsd-powered-trueos-with-its-custom-qt-desktop?p=896691#post896691 recalls:
… top notch. It graphical program that can send ZFS snapshots to a FreeNAS server, and also restore your system from boot CD from said ZFS snapshots. That is a really impressive program and they get a WELL DONE TRUEOS! for that.
☑
Some time after switching to FreeBSD-CURRENT I did install the port and whilst I always intended to begin using the software, I never did because I couldn't put my mind to the security aspect, which was relatively difficult (for me) without the TrueOS Desktop context.
A handful of other links, for posterity:
- The Revamped Life-Preserver - iXsystems, Inc. - Enterprise Storage & Servers (2013-10-31)
- PC-BSD | History of PC-BSD | Ideal for desktop or server (Wayback Machine, 2015-12-08)
- Backup utility as mindless as OS X Time Machine? | The FreeBSD Forums (2016-11-18)
- LinuxAndUbuntu Review Of TrueOS A Unix Based OS – Life Preserver (2019-12-13)
Backup options … myriad, but I never read of anything, for FreeBSD, that came close to Apple's Time Machine experience.
Life Preserver was closest, to the best of my knowledge, but (without TrueOS Desktop) there was the starting hurdle.
Speeding through FreshPorts today, this caught my eye:
sysutils/py-zfs-autobackup – ZFS autobackup
Periodically backup ZFS filesystems to other locations
https://github.com/psy0rz/zfs_autobackup#readme
… and so on.
If Life Preserver can't be suitably resurrected then ZFS autobackup, or something like it, might form the basis of an alternative.
If I understand it correctly, Life Perserver "only" automates the automatic scheduled taking and removal of local ZFS snapshots but does not help you to get those backed up e.g., to the Cloud. Or does it?
Features of Life Preserver included automated send/receive with e.g. FreeNAS. In the screenshot above there's the cloud icon with an up-arrow; the Replication button.
Defocusing from TrueNAS
https://www.google.com/search?q=ZFS+send%2Freceive+cloud&tbs=li%3A1#unfucked&tbs=qdr:y ▶ rsync.net: ZFS Replication to the cloud is finally here—and it’s fast | Ars Technica (2015-12-17)
Constraining the search results to the past year ▶ a variety of discussions, I don't keep abreast of ZFS-friendly cloud offerings but for one of the first search results that I randomly picked – recent Cloud backup solution that is ZFS-friendly : zfs on Reddit – rsync.net was the (still) the first answer to a question.
https://www.rsync.net/pricing.html: Interesting, but pricing starts with a 400 GB Minimum Order at 10 USD per month, which is prohibitive if you want to store e.g., only ~/.config
Yeah I'm not recommending any particular product or service (we have backup discussions galore elsewhere on the Internet), just focusing on ZFS and the only snapshot-oriented app that was memorable to me.
I imagine that we'll also find web app interfaces to ZFS snapshots, send and receive.
Food for thought, via https://old.reddit.com/r/zfs/comments/kwo8t1/recursive_mount_of_snapshots/gj94q6e/:
https://github.com/michabbs/rclone-sync-zfs
Backup strategy idea: Restic for fast backups & restores https://people.freebsd.org/~dch/posts/2021-03-05-restic-backups.html
Idea: Instead of pruning old backups periodically, why not prune the oldest ones at the time directly after new ones have been made, and show the process as "complete" to the user only once the pruning is complete?
Started a Backup.app that (for now) just saves the zroot/var/home dataset in a backup.zfs file.
Could/should be expanded to
- Add restoring from backups
- Add optional backup encryption
- Use a zfs pool instead of a
.zfsfile - Do incremental backups
- Do backups over the network, ideally just requiring
ssh(so one could do ZFS based cloud backups?) - Maybe support the most popular FreeBSD based network backup server types (TrueNAS?) ideally autodiscovered with Zeroconf (if it supports that)
- ...
Anyone wants to help me make it?
The beginnings are at https://github.com/helloSystem/Utilities/tree/master/Under%20Construction/Backup.app