Recommended strategy for Multi Master Sync?

[ryanerwin]

trafficstars

I want to use gocryptfs to store credentials that I synced across multiple systems, where the new writes/updates could come from either direction. The most important thing is that when their are conflicts, it would be far better to manually delete an old file that is out of date than to mistakenly ignore an update...

Curious if anybody is using gocryptfs this way and if there's something better than running git or rsync on top as a way to keep these files in sync between multiple hosts.

[bexelbie]

How are you planning to resynchronize the other hosts when one changes something? I suspect the right answer is to either make sure you have pulled down the latest modifications well before you make new ones and push an update or to use an alternative system.

Since you're storing credentials, it feels like a better solution might be something like pass on top of git where the credential files are individually encrypted and maintained. gocryptfs focuses on encryption of the entire directory structure, effectively as a blob, aiui.

[ryanerwin]

Thanks for your ideas @bexelbie

How are you planning to resynchronize the other hosts when one changes something?

I was thinking of using Unison, SyncThing or FileSync to keep the files in sync... Basically one "server" that all of the clients would sync to, similar to my dotfiles.

I suspect the right answer is to either make sure you have pulled down the latest modifications well before you make new ones and push an update

That's helpful. Perhaps it will be best to just write some simple git wrapper scripts, or maybe even put something more interesting together with go-git on top of gocryptfs.

Since you're storing credentials, it feels like a better solution might be something like pass on top of git where the credential files are individually encrypted and maintained

I was thinking more about ~/.ssh and ~/.aws, where the credentials are files or embedded in files.

Even for more traditional private documents like Passport scans though, I would definitely want to sync everything inside gocryptfs between my desktop, laptop, and phone, and allow for changes to be made at any of those locations.

[bexelbie]

Thanks for your ideas @bexelbie

How are you planning to resynchronize the other hosts when one changes something?

I was thinking of using Unison, SyncThing or FileSync to keep the files in sync... Basically one "server" that all of the clients would sync to, similar to my dotfiles.

Consider reading about the use of gocryptfs and Dropbox as it seems like the same basic use concept. I don't do this, but I recall it being well documented.

I suspect the right answer is to either make sure you have pulled down the latest modifications well before you make new ones and push an update

That's helpful. Perhaps it will be best to just write some simple git wrapper scripts, or maybe even put something more interesting together with go-git on top of gocryptfs.

git on top of gocryptfs (meaning managing the gocryptfs files) just feels weird to me. AIUI, gocryptfs does deduplication and maintains indexes. You don't get "change" level granularity that will make sense for merges/conflicts. I get the feeling this is not the right tool for this use case.

Since you're storing credentials, it feels like a better solution might be something like pass on top of git where the credential files are individually encrypted and maintained

I was thinking more about ~/.ssh and ~/.aws, where the credentials are files or embedded in files.

Even for more traditional private documents like Passport scans though, I would definitely want to sync everything inside gocryptfs between my desktop, laptop, and phone, and allow for changes to be made at any of those locations.

I use gocryptfs for something like this (dot-files, not other files - but there is no reason it couldn't be for all files). I do not use it to keep multiple working location is sync though. If you're interested it is here: https://github.com/bexelbie/bexelbie-ansible-playbook

[impredicative]

Just for completeness, gec assists in running gocryptfs in git. As such, syncing the crypt dir across multiple nodes is trivial with it, and also without it. This comes with a number of caveats, and it then helps if any routinely modified files are kept small. Yes, it's not the "right tool" for this problem, but until we have the right tool, it works reasonably.