git: add jj git sync command

[0xdeafbeef]

Checklist

If applicable:

• [x] I have updated CHANGELOG.md
• [ ] I have updated the documentation (README.md, docs/, demos/)
• [ ] I have updated the config schema (cli/src/config-schema.json)
• [x] I have added/updated tests to cover my changes

Closes #1039

[0xdeafbeef]

Wrote it according https://github.com/jj-vcs/jj/issues/1039#issuecomment-2465742400.

[0xdeafbeef]

Status report - i'll try to finish it on this weekend. Daily driving it, found a nasty bug :)

[0xdeafbeef]

@yuja

I think we need to use the local bookmark tracking the remote. Remote bookmarks often point to hidden commits.

It didn't work well, so i've a hybrid solution.

The problem which i debuged for so long was when 2 bookmarks pointed to the same comit and second bookmark moved in remote. Sync will move my local bookmark to the same position.

Now daily drive it for a week, no issues so far.

[yuja]

I think we need to use the local bookmark tracking the remote. Remote bookmarks often point to hidden commits.

It didn't work well, so i've a hybrid solution.

The problem which i debuged for so long was when 2 bookmarks pointed to the same commit and second bookmark moved in remote. Sync will move my local bookmark to the same position.

Hmm, I'm not sure how it's related to whether to use local/remote bookmarks, but that sounds good. afaiui, sync should simulate the movement that would occur when remote changes were made locally. If the same commit were rewritten locally, these two bookmarks would be moved to the new commit.

For tracked remote bookmarks, the latest local states are represented by the tracking local bookmarks, so I think it makes sense to use the corresponding local bookmarks. For untracked remote bookmarks, we'll need to use remote bookmarks.

EDIT: That said, if the same change were rewritten concurrently, the result should be divergence. So I'm getting less sure about that. It might be safer to leave descendants unchanged if the unsynced tracked remote bookmarks were rewritten at the remote.

[jgilchrist]

@0xdeafbeef I've tried this out and unless I'm misunderstanding, it doesn't seem to be working as expected for me. Prior to running jj git sync things look like:

i.e. I've got a few changes, some with bookmarks, all based on main. main@upstream has a single new commit, so I'm expecting that after the sync, all of these changes are rebased. However, after jj git sync, I see:

i.e. all of the WIP changes are hidden, with the working copy now on top of the new main (synced with main@upstream) and with all of the bookmarks now pointing to that commit. The output of the sync shows:

It looks like we do attempt to rebase the changes ('Rebasing 7 commits') but then they are somehow not rebased successfully ('7 already merged').

Any idea what might be going wrong here?

[zx8]

Please don't let this PR become stale! I'm super excited to see it merged - is there anything you need in terms of testing/data to get this over the line @0xdeafbeef? 🙏

[0xdeafbeef]

Please don't let this PR become stale! I'm super excited to see it merged - is there anything you need in terms of testing/data to get this over the line @0xdeafbeef? 🙏

Example of @jgilchrist showed that my impl is unsound and works only in basic cases. I'll try to fix it in the next 2 days and push updated version. Sync won't have merge support cause this rabbit hole appears to be deep enough :)

[0xdeafbeef]

@0xdeafbeef I've tried this out and unless I'm misunderstanding, it doesn't seem to be working as expected for me. Prior to running jj git sync things look like:

i.e. I've got a few changes, some with bookmarks, all based on main. main@upstream has a single new commit, so I'm expecting that after the sync, all of these changes are rebased. However, after jj git sync, I see:

i.e. all of the WIP changes are hidden, with the working copy now on top of the new main (synced with main@upstream) and with all of the bookmarks now pointing to that commit. The output of the sync shows:

It looks like we do attempt to rebase the changes ('Rebasing 7 commits') but then they are somehow not rebased successfully ('7 already merged').

Any idea what might be going wrong here?

strangely that i've not caught this during my daily-driving.

Problem was that instead of

  Before sync:                After sync:

      C---D---E (local)                  C'--D'--E' (local rebased)
     /                                  /
    A---B (origin)             A---B---F (origin with new commit F)

I was doing this:

     Before sync:                After sync:

      C---D---E (local)          [commits C,D,E abandoned]
     /
    A---B (origin)             A---B---F (origin, all bookmarks collapsed here)

Because of incorrect api usage. It worked fine with a single  feature branch, but moved branches to the same commit in case of several branches, which caused it to think that these commits are duplicate for F and should be abandoned.

[jgilchrist]

Thanks @0xdeafbeef! Tried this again with a similar setup and my changes get rebased as I expected 👍

However some other changes also got rebased too. In my case it looks like everything in descendants(master@upstream) got rebased, but this included immutable changes that were on untracked remote feature branches, i.e.:

Before:

gitGraph
    commit id: "main"
    branch local
    checkout local
    commit id: "a"
    commit id: "b"
    checkout main
    branch "another@upstream"
    commit id: "c"

After (expected)

gitGraph
    commit id: "oldmain"
    branch "another@upstream"
    commit id: "c"
    checkout main
    commit id: "main"
    branch local
    checkout local
    commit id: "a'"
    commit id: "b'"

After (actual)

gitGraph
    commit id: "oldmain"
    checkout main
    commit id: "main"
    branch local
    checkout local
    commit id: "a'"
    commit id: "b'"
    checkout main
    branch "another@upstream"
    commit id: "c'"

[0xdeafbeef]

Thanks @0xdeafbeef! Tried this again with a similar setup and my changes get rebased as I expected 👍

However some other changes also got rebased too. In my case it looks like everything in descendants(master@upstream) got rebased, but this included immutable changes that were on untracked remote feature branches, i.e.:

Before:

gitGraph
    commit id: "main"
    branch local
    checkout local
    commit id: "a"
    commit id: "b"
    checkout main
    branch "another@upstream"
    commit id: "c"

After (expected)

gitGraph
    commit id: "oldmain"
    branch "another@upstream"
    commit id: "c"
    checkout main
    commit id: "main"
    branch local
    checkout local
    commit id: "a'"
    commit id: "b'"

After (actual)

gitGraph
    commit id: "oldmain"
    checkout main
    commit id: "main"
    branch local
    checkout local
    commit id: "a'"
    commit id: "b'"
    checkout main
    branch "another@upstream"
    commit id: "c'"

yep:( . already working on it. When we rebase master on top of master@remote we automatically rebase all feature branches branched from master, which causes big mess

[0xdeafbeef]

@jgilchrist i've updated it

[0xdeafbeef]

@yuja @PhilipMetzger I've changed how sync works because the previous design had serious flaws - when syncing main it would rebase ALL its descendants.

Now we use fork-point rebasing for each changed bookmark, equivalent to jj rebase --branch bookmark --destination bookmark@remote This only rebases commits that actually diverged from the remote, not all descendants.

Do you have any comments for it?

I'll refactor and rebase onto main after you approve/disapprove design changes

[yuja]

Now we use fork-point rebasing for each changed bookmark, equivalent to jj rebase --branch bookmark --destination bookmark@remote

I haven't reviewed the implementation thoroughly, but does it rebase anonymous branches?

                      B'
B                     |
|               =>    A' foo@origin
A foo@origin       A

The condition to rebase might be something like old_base..head & ::untracked_remote_bookmarks() == none().

BTW, I expect the core logic and tests will be moved to jj-lib.

[zx8]

@0xdeafbeef I'll refactor and rebase onto main after you approve/disapprove design changes

Is there anything you need from the jj maintainers to get this merged, or is it simply a matter of finding the time to work on it? 🙏

[0xdeafbeef]

@0xdeafbeef I'll refactor and rebase onto main after you approve/disapprove design changes

Is there anything you need from the jj maintainers to get this merged, or is it simply a matter of finding the time to work on it? 🙏

problem is the lack of time and that it's not so trivial as i thought :) I'll rebase on main today and try to resolve all comments