redu icon indicating copy to clipboard operation
redu copied to clipboard

POC: optionally use rustic_core

Open aawsome opened this issue 9 months ago • 3 comments

Here is a working POC which optionally uses rustic_core, see #87.

I think there are several things to enhance/fix, but I'll leave that to you @drdo ;-)

Here are the points, I stumbled over in the PoC:

  • error handling has been done just quick&dirty using anyhow or sometimes unhandled
  • Configuration of repository (storage+password): Here rustic has a complete different philosophy as restic and prefers config files to do the configuration. For the POC only local storage and a entered password works, see the unimplemented!() in the code and the docu to BackendOptions and RepositoryOptions in rustic_core. Actually if you were only using rustic_core, I would suggest to you to use those as leading config structs. Currently for the only working local storage, there is also way too much dependency overhead which is mostly because of the many storage backends rustic_core supports via opendal.
  • Due to the design (which I didn't want to break), for each snapshots and ls call, rustic (re-)opens the repository and (re-)reads the index. If you would change the design, things could work with just a single opening and index-load which would give much better performance when reading the tree from multiple snapshots.
  • I realized that you use all snapshot data to identify snapshots - snap.id would suffice. I also just mapped the needed data, feel free to complement the mapping from rustic_core::SnapshotFile to redu::Snapshot

aawsome avatar Apr 17 '25 10:04 aawsome

Now the repository is only opened once and index is only read once (in Rustic::new()) which gives an immense performance boost especially when scanning multiple snapshots tree. It is still not optimal - better would be to not read the index for config() and snapshots() (not needed there) and only read it once before all ls() calls (and don't read it at all if no ls() is called) - but that would require a structural change outside of the restic module - feel free to do that yourself!

aawsome avatar Apr 18 '25 04:04 aawsome

Is there something in rustic to parse repo strings in a restic-compatible way? For example an sftp repo.

drdo avatar Apr 29 '25 18:04 drdo

Is there something in rustic to parse repo strings in a restic-compatible way? For example an sftp repo.

BackendOptions::to_backends() already does string parsing; however only local, rclone, rest and opendal are supported. To use a backend supported by opendal, you need to give opendal:<SERVICE> as repo string (e.g. opendal:sftp) and supply options (like server, username etc) in BackendOptions::options

aawsome avatar Apr 29 '25 19:04 aawsome