AMP
AMP copied to clipboard
CubeCoders
Back-up directly to S3 instead of locally + copy to S3
Feature Request
Feature Information:
I would like to keep my minecraft VM as weight-light as possible (as far that's possible with minecraft). For backups I use minIO to self-host s3 buckets on my NAS. Every time a backup is made, it is stored on the VM and then copied to the NAS. For me it's not needed to keep the backup on the local machine, and it would be better to automatically delete the backup after a successful upload to S3.
This is for the Minecraft module btw. I am not sure how the other modules look like, maybe it's a good addition to these as well.
I think a good solution could be something like this:
Note that the local backup should only be deleted if the upload to S3 has been successful! (maybe consider a retry mechanism and an extra option with how many retries should be attempted, and how much time should be in between each retry)
Would also be very cool to backup directly to S3 without first making a local copy, but I don't know if that's possible.
TLDR
Add an option to automatically delete backups after they have been successfully uploaded to the S3 bucket
I confirm:
- [x] that I have searched for an existing feature request matching the description.
Hi @PhonicUK ,
I wanted to add a suggestion to this.
Would it be possible to have incremental backups in AMP? This will mean we don't need terabytes to store a month worth of backups.
Awesome to hear! I might have some spare time early 2024. If you need any help let me know. (as far as the company allows third party contributions.)
From: PhonicUK @.> Sent: Wednesday, September 20, 2023 11:17:32 AM To: CubeCoders/AMP @.> Cc: Joost R. @.>; Author @.> Subject: Re: [CubeCoders/AMP] Back-up directly to S3 instead of locally + copy to S3 (Issue #917)
It is planned but really tricky to do and get right.
— Reply to this email directly, view it on GitHubhttps://github.com/CubeCoders/AMP/issues/917#issuecomment-1727316292, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFAMG55B64TBKYSKBAQICSTX3KYCZANCNFSM6AAAAAA32NRG6Y. You are receiving this because you authored the thread.Message ID: @.***>
I second this feature request!
I too have my own S3 and would love to have it delete local copies after it uploads to S3 successfully.
I was at least able to set some kind of backup rotation through "performance and limits" with the replacement policy and max backup size limits.
I was at least able to set some kind of backup rotation through "performance and limits" with the replacement policy and max backup size limits.
Same. Now it keeps one backup locally at all times, and when the next backup is due, it deletes that one, makes a new one, and uploads that one. However just deleting it after upload would be better, and directly backing up to s3 without making a local backup would be best, but probably very hard if not impossible in zip format.
So does the backup count limit only affect the local backups and not S3? If so, in theory I can set the limit to 1 and the S3 files are untouched?
Yes that's what I do indeed. Amp only uploads to s3, it does never delete. That's a manual process.
From: FiberGeek @.> Sent: Tuesday, January 23, 2024 10:40:44 PM To: CubeCoders/AMP @.> Cc: Joost R. @.>; Author @.> Subject: Re: [CubeCoders/AMP] Back-up directly to S3 instead of locally + copy to S3 (Issue #917)
So does the backup count limit only affect the local backups and not S3? If so, in theory I can set the limit to 1 and the S3 files are untouched?
— Reply to this email directly, view it on GitHubhttps://github.com/CubeCoders/AMP/issues/917#issuecomment-1906965176, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFAMG57CYBWSQ73V2XA3F2TYQAU5ZAVCNFSM6AAAAAA32NRG62VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBWHE3DKMJXGY. You are receiving this because you authored the thread.Message ID: @.***>