gb icon indicating copy to clipboard operation
gb copied to clipboard

weird flow when restoring a removed directory

Open alive opened this issue 2 years ago • 12 comments

When trying to restore a directory with a non-null files.end, it is pretty much a certainty that files.end < now. I don't understand why gb fails to restore instead of just restoring latest version? This issue is compounded because as far as I can tell there's not a clear way to get the latest timestamp for a directory? I had to gb ls and grab a random one from a file, though this may just be me missing something.

To resolve, I think:

  • gb restore should not exclude files where files.end < now in the query
  • gb ls should also output information about directories (though this seems much more complicated)

What do you think?

alive avatar Jan 16 '23 10:01 alive

To be clear, this is the error I'm talking about panic: {path} did not exist in the database (as either a file or directory) as of that timestamp

alive avatar Jan 16 '23 10:01 alive

Wouldn't it be surprising if you restored a directory and it gave you every file that's ever been in that directory? How would that work if there were many revisions of the same file (with the same name)? I think it's correct to ask you for a timestamp and restore the directory as it was at that timestamp.

leijurv avatar Jan 16 '23 19:01 leijurv

Also I'm shocked that someone is using gb! LOL

How long have you been using it?? with how many files?

leijurv avatar Jan 16 '23 19:01 leijurv

i also use gb

5HT2 avatar Jan 16 '23 20:01 5HT2

really???????????????????????????

leijurv avatar Jan 16 '23 21:01 leijurv

why

leijurv avatar Jan 16 '23 21:01 leijurv

i didn't like how anything else worked and decided "well if it's in Go and i don't like smth i can hack together my own features" and ended up being okay with how it works.

normally i'd just roll my own solution for shit but i don't have the time anymore to write my own backup solution and test it in real world senarios to be sure it's reliable so i trust yours works fine rofl

~150k files stored as of this morning :p

5HT2 avatar Jan 16 '23 22:01 5HT2

Wouldn't it be surprising if you restored a directory and it gave you every file that's ever been in that directory? How would that work if there were many revisions of the same file (with the same name)? I think it's correct to ask you for a timestamp and restore the directory as it was at that timestamp.

This is a good point, maybe it should restore all files with timestamp = max(timestamp over all files in directory)? My main concern is that it is not so clear what timestamp I should be restoring to in the case where I want the latest files?

Also I'm shocked that someone is using gb! LOL

How long have you been using it?? with how many files?

For about 8 months, and with only about 10gb of files (I don't backup much media with gb)

alive avatar Jan 16 '23 22:01 alive

Wow, cool! What storage do you guys use? S3 (the actual S3 or a compatible API?) or gdrive?

This means gb has at least 5 users! Me, @babbaj @glen3b and you two! That's so exciting!

leijurv avatar Jan 17 '23 18:01 leijurv

My main concern is that it is not so clear what timestamp I should be restoring to in the case where I want the latest files?

This is a reasonable idea. I'd be okay with adding an option to restore to do the most recent version of every file, but it won't be the default.

leijurv avatar Jan 17 '23 18:01 leijurv

Actual S3 in my case :3

5HT2 avatar Jan 17 '23 19:01 5HT2

wip here https://github.com/leijurv/gb/pull/28

alive avatar Jan 19 '23 04:01 alive