External conflict handling tool (to merge, or select winner)

[napseis]

hi,

Now that syncthing has a conflict handling system, it thought would be nice to propose some kind of semi-automatic conflict resolution. (especially on first time merge, despite i saw the temporary master trick was a nice idea on some other thread) I was thinking if something like git merge tool would be possible? That is, syncthing knows the conflict file list, and open something like vimdiff, meld, etc, to ask the user to resolve conflicts one after the other ?

Thank you

[calmh]

That would be awfully file type dependent; merging jpegs or word documents would suck, using vimdiff. :)

[napseis]

yes, indeed, maybe I was too specific, sorry. More generally, despite I have now idea how dificult it would be, it still would be usefull to have in one place the list of files that were declared to be in conflict ? So that, right after a sync, you know what needs to be taken care of ?

[calmh]

More than simply something like find . -name \*sync-conflict\*?

[tomtom]

Unison allows users to define a mergetool for files matching a specific pattern. Something like meld is extremely useful in such a situation since those kind of manually resolvable usually happen with text files. Calling meld on all files with unresolved sync conflicts would be of great help IMHO.

[calmh]

Meld looks like some interactive merge tool? How would syncthing use it to resolve conflicts? Or are you suggesting running an external tool of some sort (meld) to automatically resolve conflicts when they happen, basing success on exit code? That would probably be at least technically feasible.

[tomtom]

Meld looks like some interactive merge tool? How would syncthing use it to resolve conflicts?

Let me clarify my point of view. There is a point where sync conflicts cannot be resolved automatically by syncthing and where users have to intervene manually. This is where unison uses a mergetool. It would be nice if syncthing would support users to call an interactive merge tool on files for which sync conflicts cannot be resolved automatically.

[tomtom]

Meld looks like some interactive merge tool? How would syncthing use it to resolve conflicts?

Note: unison allows users to use any merge tool. diff3 does automatic merges if possible. meld is an interactive merge tool.

The OP asked to call such a merge tool in case of conflicts. You replied this is type dependent. My point was that unison allows users to use a merge tool per file pattern. Examples:

 merge = Regex \.txt -> meld CURRENTARCH CURRENT1 CURRENT2 && cp 

CURRENT1 NEW1 merge = Name * -> diff3 -m CURRENT1 CURRENTARCH CURRENT2 > NEW

Syncthing could provide a similar mechanism to invoke a user defined merge tool in case of conflicts.

[calmh]

We could, but not an interactive one. But i could see running a command for conflict handling, much like the external versioning. If that command merges, picks the newest file or just tosses dice is then not our concern.

[tomtom]

I understand your point but in my experience, automatic merging really hurts in 0,1% of the cases, which eventually made me switch to an interactive merge tool. Anyway, I now run unison before & after running syncthing which also solves the problem with symlinks for me.

[Stebalien]

See ficus for related work: http://nil.csail.mit.edu/6.824/2015/papers/ficus.pdf

[calmh]

The fact that there's a 20 year old paper on implementing it in distributed file systems, but that noone notable has actually done that since speaks volumes to me. ;)

[joopdo]

Although dated post, I 'fixed' my conflicts with a simple bash script. https://github.com/joopdorresteijn/dotfiles/blob/master/scripts/syncconflict.sh

Now to solve the origional cause in the settings of syncthing, perhaps I need a foldermaster setup.

[xeruf]

I would propose to have a setting on how to resolve conflicts with three options:

1. Create conflict-files (as currently)
2. Only keep the newer version (useful if the folder consists of non-sensitive data - e.g. I might get conflicts because I edit the tags of my music files on two computers, but resolving a conflict would be much more hassle than simply updating them again)
3. Run a script. It would be run with paths of the base version and the two conflicting alternatives. It could then consider the file format, timestamps and content and thus carry out simple three-way merges for text files, keep only the latest version for binaries (i.e. selectively implement option 2) or even copy the information somewhere so it could later be used in a graphical merge tool. Syncthing runs and awaits the script - if it exits with zero, the conflict is deemed resolved, if it is non-zero, Syncthing creates a conflict-file as usual.

[calmh]

For the record, option 2 exists in the maxConflicts=0 setting.

[rrouviere]

Hi, I just want to bring up a potential use case for option 3.

I currently use syncthing to synchronise a keepass database between my phone and a pc. Having the db unlocked on one device while using it on another create a conflit, as does a number of other operations. As such, conflits occurs quite frequently, which is expected. Fortunately, it is quite easy to merge two db together, as keepass has an integrated merge feature.

However, being able to provide syncthing with a script to execute to solve conflits would greatly ease the process, since keepassxc-cli has an option to automatically merge two databases.

Using the script exit code as @xeruf suggested would allow for scripts to solve simple cases while not doing any harm in the 0,1% cases that tomtom mentionned :).

However, I think that it would be preferable for syncthing not to wait on the script, as that would allow for interactive scripts too

[xeruf]

However, I think that it would be preferable for syncthing not to wait on the script, as that would allow for interactive scripts too

I have no idea how you want to make interactive scripts work for a background process.

Syncthing should definitely wait on it - if you want to have something long-running, you can fork off and provide an appropriate exit code.

[rrouviere]

Syncthing should definitely wait on it - if you want to have something long-running, you can fork off and provide an appropriate exit code. Providing an exit code before the long-running process ends would mean choosing the "right" version without waiting for the long-running process, in which case what's the use case for it anyway ?

I have no idea how you want to make interactive scripts work for a background process.

Fair point, what I mean by "not waiting" is that others shares should not if possible by impacted by either a long-running or interactive script.

[solarkraft]

Is anyone considering implementing this (non-interactively for now)?

It could look a lot like external versioning, with a "conflict handing" panel added to a folder's settings and preferably some template-able command that could supply the two files in conflict and the common ancestor so that you can do a three-way-merge.

[xeruf]

I have implemented something like this myself in https://git.jfischer.org:444/xeruf/synct/src/branch/main/synct-diff