AMP
AMP copied to clipboard
CubeCoders
Differential backups (perhaps integrate ZFS Snapshots).
I confirm that
- [x] I have searched for an existing feature request matching the description.
Feature Description
I've just had to delete a bunch of backups of my Minecraft server, as I had hit the allocated storage limit. In part because within the space of a couple of weeks of solo play, my server has ballooned to 2.5GB, and because each backup is a complete copy. I could increase the allocated storage but the thought did occur to me that "gee, I wish I could just use ZFS's Snapshot feature". So rather than 10 backups taking up my entire storage allocation, I could have literally months of hour by hour snapshots taking up the same amount of space. The reason being is that ZFS Snapshots can be differential, so that the Tuesday's backup is only larger than Monday's backup, by the amount of new data generated from one day to the next. So it'd be 2.5GB + 10MB say.
It would have the added benefit for Linux / ZFS users like myself in that it would eliminate some duplicated effort and unnecessary storage use. I already have everything on my server's storage pool automatically snapshotted by ZFS. So when AMP creates a new backup and ZFS takes a new snapshot, that snapshot is uses more storage than it would need to, if AMP was able to use ZFS Snapshots.
In all fairness, I could just disable AMP's backup feature and set up the Snapshots myself but not only do I not know whether AMP would act up if I were restore a Minecraft server by rolling back a Snapshot, it would also be nice to have it integrated into AMP's interface.
don't know if it works/you tried it. but used Config -> Backup -> Use Smart Backup Exclusion?
Thanks for the suggestion @Psychloor.
I am already using it but if I understand that setting correctly. Then the space it saves is comparatively minimal unless you're using some kind of usually large version of Minecraft.
If I go back to my example of my own Minecraft world, it's currently at 3GB for each backup. Let's say that 1GB of that would be excluded from each backup and that between each backup, my world grew by 1GB until it reached 101GB in size. With AMP's built-in backups, that would take-up about 5051GB of storage.
With ZFS Snapshots, it'd take up the 3GB that the world occupied when the first snapshot occurred. Then each subsequent snapshots would take-up only as much space as there was newly generated data within the world. Which means a grand total of 101GB*.
*In all fairness this is very much back of a napkin maths. It's based on the premise that the only new data that is created, is created by exploring the world. In other words, if 1GB worth of chunks are newly rendered between each Snapshot, then each subsequent Snapshot would require just 1GB more of disk space. In reality, storage capacity is used by a Snapshot anytime data changes too. So for example, if you Snapshot something that is constantly changing, such as a cache, then storage can be eaten up quicker... but not as quick as simply creating a copy of the entire world folder.
The actual plan is differential backups. There's no need to make it ZFS specific at all.
That's a very good point. I agree that differential backups do not have to be specific to ZFS.
If they weren't ZFS specific, then that would benefit all users. Even those running Windows. I would argue though, that ZFS is a tried and enterprise tested, existing solution to the problem.
I guess the ideal solution would be having differential backups integrated into AMP, and taking reasonable steps with AMP to ensure that it doesn't complain in the event that a users restores an Instance via a ZFS Snapshot.
