Bookmark Refs Reset After Push Due to Mismatch in .git/config Fetch Setting

[tim-janik]

Description

I'm seeing an issue where a bookmark ref is being reset to a previous commit after a push. This behavior seems to be related to the .git/config file's fetch settings. Specifically, when the fetch setting is not +refs/heads/*:refs/remotes/origin/*, the updated remote ref is not immediately fetched after push, causing the next jj status to reset it.

Steps to Reproduce the Problem

1. Clone a colocated repository, e.g. jj
2. Ensure the .git/config file has a fetch setting that is not +refs/heads/*:refs/remotes/origin/*. For example:

   [remote "origin"]
       url = [email protected]:martinvonz/jj.git
       fetch = +refs/heads/main:refs/remotes/origin/main
   

3. Set a bookmark to a new commit.
4. Push the new bookmark to the remote.
5. Run jj status after the push.
6. Observe that the bookmark ref is reset to a previous commit.

Expected Behavior

The bookmark ref should not be reset at jj status after push. The updated remote ref should be immediately fetched and reflected in the local state.

Actual Behavior

The bookmark ref is not fetched after push, which can be proven by looking at the local git refs. This cannot be reproduced if the .git/config file contains +refs/heads/*:refs/remotes/origin/*

Additional Information

According to the documentation, the fetch setting in .git/config is ignored by libgit2 and therefore jj. However, this is not the case, and the fetch setting seems to affect the push behavior.

Logs:

$ jj git push -b tim-janik/push-rtpllwlqomtn
Changes to push to origin:
  Move sideways bookmark tim-janik/push-rtpllwlqomtn from ba9a6eeb584d to 28a06d5a9151
$ find . -name \*rtpllwl*
./.git/refs/remotes/origin/tim-janik/push-rtpllwlqomtn
./.git/refs/heads/tim-janik/push-rtpllwlqomtn
./.git/logs/refs/remotes/origin/tim-janik/push-rtpllwlqomtn
./.git/logs/refs/heads/tim-janik/push-rtpllwlqomtn
$ tail ./.git/refs/remotes/origin/tim-janik/push-rtpllwlqomtn ./.git/refs/heads/tim-janik/push-rtpllwlqomtn ./.git/logs/refs/remotes/origin/tim-janik/push-rtpllwlqomtn ./.git/logs/refs/heads/tim-janik/push-rtpllwlqomtn
==> ./.git/refs/remotes/origin/tim-janik/push-rtpllwlqomtn <==
ba9a6eeb584dc8390433b130e3d0342fcf30e671

==> ./.git/refs/heads/tim-janik/push-rtpllwlqomtn <==
28a06d5a915118c9178d2ae52da16515b0c4db86
==> ./.git/logs/refs/remotes/origin/tim-janik/push-rtpllwlqomtn <==
0000000000000000000000000000000000000000 c26b7ce8bc0635462ce4a4b4157e7ed8e80b4908 unknown <unknown> 1734377420 +0100
c26b7ce8bc0635462ce4a4b4157e7ed8e80b4908 ba9a6eeb584dc8390433b130e3d0342fcf30e671 unknown <unknown> 1734378979 +0100

==> ./.git/logs/refs/heads/tim-janik/push-rtpllwlqomtn <==
0000000000000000000000000000000000000000 c26b7ce8bc0635462ce4a4b4157e7ed8e80b4908 Tim Janik <[email protected]> 1734377536 +0100	export from jj
c26b7ce8bc0635462ce4a4b4157e7ed8e80b4908 f50bcf4b2c3f30edce6002b40161b5cb56fb1f99 Tim Janik <[email protected]> 1734377863 +0100	export from jj
f50bcf4b2c3f30edce6002b40161b5cb56fb1f99 ba9a6eeb584dc8390433b130e3d0342fcf30e671 Tim Janik <[email protected]> 1734377874 +0100	export from jj
ba9a6eeb584dc8390433b130e3d0342fcf30e671 c26b7ce8bc0635462ce4a4b4157e7ed8e80b4908 Tim Janik <[email protected]> 1734377975 +0100	export from jj
c26b7ce8bc0635462ce4a4b4157e7ed8e80b4908 ba9a6eeb584dc8390433b130e3d0342fcf30e671 Tim Janik <[email protected]> 1734378979 +0100	export from jj
ba9a6eeb584dc8390433b130e3d0342fcf30e671 28a06d5a915118c9178d2ae52da16515b0c4db86 Tim Janik <[email protected]> 1734379059 +0100	export from jj
$ jj st
Abandoned 1 commits that are no longer reachable.
Done importing changes from the underlying Git repo.
The working copy is clean
Working copy : oyvkwkkn 8ca8f018 (empty) (no description set)
Parent commit: ryxozuxo fca92f1e cli: add --config-file=PATH argument

Notice the difference between .git/refs/remotes/origin/tim-janik/push-rtpllwlqomtn and .git/refs/heads/tim-janik/push-rtpllwlqomtn right after push.

The suggested fix is to ensure that pushed bookmarks are properly fetched as part of push, regardless of any fetch settings in the Git configuration.

Specifications

• Platform: Linux
• Version: jj-0.24.0 + a few upstream commits, I am pretty sure I have seen this with 0.23.0 as well.

[yuja]

Yes, the current Git integration assumes that the remote refs are moved accordingly. Maybe there would be some way to force libgit2 do the default thing.

[tim-janik]

Note that it is very easy to run into the described situation when people clone a shallow repo:

$ git clone --depth=1 [email protected]:martinvonz/jj.git shallow-jj
Cloning into 'shallow-jj'...
remote: Enumerating objects: 499, done.
remote: Counting objects: 100% (499/499), done.
remote: Compressing objects: 100% (441/441), done.
remote: Total 499 (delta 89), reused 188 (delta 44), pack-reused 0 (from 0)
Receiving objects: 100% (499/499), 1.79 MiB | 4.76 MiB/s, done.
Resolving deltas: 100% (89/89), done.

$ cat shallow-jj/.git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = [email protected]:martinvonz/jj.git
	fetch = +refs/heads/main:refs/remotes/origin/main
[branch "main"]
	remote = origin
	merge = refs/heads/main

[isovector]

~Is there a fix for this problem? It's been driving me crazy all week.~

-	fetch = +refs/heads/main:refs/remotes/origin/main
+	fetch = +refs/heads/*:refs/remotes/origin/*

fixed the issue

[martinvonz]

Yes, the current Git integration assumes that the remote refs are moved accordingly. Maybe there would be some way to force libgit2 do the default thing.

I looked a bit through the libgit2 code but I couldn't find a way to tell the push code to use a specified fetch refspec. I filed https://github.com/libgit2/libgit2/issues/6969 to add support for it. It's possible I missed it and it's already supported. I suppose they'll let us know in that case.

[sfc-gh-uvarma]

A more cumbersome workaround to this issue that doesn't involve changing the fetch refspec is the following:

$ git push origin $BRANCHNAME --force-with-lease
$ jj git fetch --branch $BRANCHNAME

Do not attempt to run any other jj command between these two commands or jj will end up abandoning changes.

Personally, I have a fetch refspec that is limited to branches that I care about most of the time. For the few cases where I'm doing working outside those branches, this approach becomes necessary.

I don't know how easy it would be to detect from the jj side that the branch(es) that we pushed with jj git push were not in the fetch refspec and in those cases immediately run the equivalent of jj git fetch --branch $BRANCHNAME before attempting to get jj to "catch up" to the changes in Git.