dokploy icon indicating copy to clipboard operation
dokploy copied to clipboard

Ability to backup to more destination types

Open saidul opened this issue 1 year ago • 12 comments

What problem will this feature address?

Can not backup to FTP, SFTP, Google Drive other storage solutions.

Describe the solution you'd like

Add support for other destination types like Google Drive, One Drive, FTP, SFTP etc.

Describe alternatives you've considered

During backup destination creation, we can select destination types and their credentials accordingly. This can later be used in upload phase of backup process. Following can be used for uploading the data:

  • rclone - supports a lot of destination. destination configuration can be given as argument. Using rclone will probably be easiest and fastest to add support for many providers.
  • https://github.com/duna-oss/flystorage - common api for services. adapter based.
  • Independent libraries - will require more effort but provides granular control over each api.

Additional context

No response

saidul avatar Sep 04 '24 03:09 saidul

Sounds like a good idea, it is in my plans to come up with a way to support multiple destination sources also for backups and more, I think rclone looks like a very good suggestion as it supports many ways, I will add to the roadmap

Siumauricio avatar Sep 05 '24 07:09 Siumauricio

Partially closed by #469, we need to adjust a little bit the code in order to support more providers, but we are using RCLONE for now for S3

Siumauricio avatar Sep 22 '24 23:09 Siumauricio

Hello, are there any updates on this?

kerimovok avatar Nov 06 '24 22:11 kerimovok

@Siumauricio Thank you so much for adding rclone support. However we still can't backup to any provider other than S3.

Rclone supports configuration via:

  • Environment variables: https://rclone.org/docs/#config-file
  • Connection strings: https://rclone.org/docs/#connection-strings
  • Config file

Different providers require different parameters. If we can support passing either environment variables or a connection string to the rclone command, it would enable compatibility with a wider range of storage providers.

I imagine following dialog can be adjusted to support advanced mode. When advanced mode/custom mode is selected, other fields related to s3 will be hidden and only 1 textbox will be present to take either the connection string or the environment variables.

Image

My humble opinion is that the environment variable is easier to define. Because when json data needs to be provided in connection string, it becomes complicated to escape the string.

Some provider requires connection info to be updated time to time. For example in case of google drive, rclone automatically refresh the tokens and store that in the config file. In such case it is probably better to have a config file defined in some volume for dockploy (like traefik config files can be modified by users seprately).

So for each backup destination we can have different config files. Which can be passed to rclone during execution of the command.

saidul avatar Mar 21 '25 19:03 saidul

Would be nice if we could also drag-and-drop a .zip file in the interface to restore from that backup, as an "escape hatch", and also save a manual backup as a .zip file from the interface

maelp avatar May 04 '25 10:05 maelp

backing up docker volumes and bind mounts would be great!

bitsmyth avatar May 08 '25 15:05 bitsmyth

/bounty $50

I'm keen to see this feature implemented, as better backup destination options would be incredibly valuable. I'm adding a $50 bounty to this issue to help accelerate its development.

To clarify the scope for anyone picking this up, here are the key tasks I'd expect:

  1. Expand Destination Options: In the backup "Destinations" section, it should be possible to add and configure various new providers, specifically including:

    • Google Drive
    • OneDrive
    • FTP
    • SFTP
    • (If other major ones are easily supported by rclone, feel free to add them too, but these are the priority)
  2. Integrate Backup Functionality: The core backup process should seamlessly integrate with these new destination providers. This means:

    • When a backup is triggered, it should successfully upload to the chosen configured destination (Google Drive, FTP, etc.).
    • The existing S3 functionality should remain untouched and continue to work as expected.
    • The proposed solution using rclone seems like a very promising path to achieve broad support efficiently.

Looking forward to seeing this feature come to life!

Siumauricio avatar May 25 '25 04:05 Siumauricio

💎 $50 bounty • Dokploy

Steps to solve:

  1. Start working: Comment /attempt #416 with your implementation plan
  2. Submit work: Create a pull request including /claim #416 in the PR body to claim the bounty
  3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

❗ Important guidelines:

  • To claim a bounty, you need to provide a short demo video of your changes in your pull request
  • If anything is unclear, ask for clarification before starting as this will help avoid potential rework
  • Low quality AI PRs will not receive review and will be closed
  • Do not ask to be assigned unless you've contributed before

Thank you for contributing to Dokploy/dokploy!

Attempt Started (UTC) Solution Actions
🟢 @luffy-orf May 25, 2025, 05:00:03 AM #1948 Reward
🟢 @vishalkadam47 May 25, 2025, 05:39:21 AM WIP

algora-pbc[bot] avatar May 25 '25 04:05 algora-pbc[bot]

RE: Algora.

This is amazing. I love incentivized rewards for contributing. I work a lot in my day job, and it's hard to justify contributing to opensource (even projects I use). I hope this can make contributing more accessible to those who have to choose between food on the table, and supporting the literal backbone of their work. (please allow replies on newsposts in discord, I don't like using issues as a forum for these comments).

AlexMason avatar May 25 '25 04:05 AlexMason

Bounties are a great idea for this project.

znowfox avatar May 25 '25 04:05 znowfox

/attempt #416

luffy-orf avatar May 25 '25 05:05 luffy-orf

/attempt https://github.com/Dokploy/dokploy/issues/416

vishalkadam47 avatar May 25 '25 05:05 vishalkadam47

Great work on your PR @vishalkadam47

Could you please also consider following things for rclone?

  1. Support Crypt This basically encrypts other storage systems making them E2EE.
  2. Support for custom rclone configuration snippet - This will keep the opportunity open to use any storage backend supported by rclone.

saidul avatar Jun 02 '25 18:06 saidul

Great work on your PR @vishalkadam47

Could you please also consider following things for rclone?

  1. Support Crypt This basically encrypts other storage systems making them E2EE.
  2. Support for custom rclone configuration snippet - This will keep the opportunity open to use any storage backend supported by rclone.

Thanks, sure will take a look at Crypt and add support for more storage providers. Waiting for @Siumauricio review on https://github.com/Dokploy/dokploy/pull/1979

vishalkadam47 avatar Jun 05 '25 17:06 vishalkadam47

@vishalkadam47 Impressive attempt — very near completion. When can we see this in production?

raqibnur avatar Aug 10 '25 05:08 raqibnur