jj
jj copied to clipboard
jj-vcs
Lightweight Git Tags vs. Annotated Git Tags
Description
Git has lightweight tags and annotated tags. An annotated tag is meant for something like a release that you push out and that should usually not be moved. A lightweight tag is meant for temporarily marking something to not loose it and easily refer to it and should usually not be pushed out.
JJ considers all Git tags the same and like annotated tags. So as soon as there is a lightweight tag in a repository, the respective commit is considered immutable. And you can also not change this without following some naming convention or taking out all tags from the immutable heads.
I think lightweight Git tags should only be treated by JJ as the temporary post-it stickers they are meant to be.
It would also be nice to be able to match in revsets whether a tag is lightweight or annotated, but I guess that would be too Git-specific. :-(
Actually a lightweight Git tag is more like a JJ bookmark.
Steps to Reproduce the Problem
jj git init foo
cd foo
jj commit -m a
jj commit -m b
git tag foo @~
jj log
Expected Behavior
a not being immutable
Actual Behavior
a is immutable
Specifications
- Platform: Git Bash in Windows Terminal
- Version: 0.34.0-22900c9a9ba362efa442fed2dd4e6e1d5c22cc7a
The distinction is not as clear cut in all implementations. E.g. when using Github, it's best to not put too much emphasis on lightweight vs annotated tags.
I am using lightweight tags for nightly releases and annotated tags for regular releases in one of my projects and ran into 2 issues with Github so far:
gh release createwill create lightweight tags (might be the same for the web interface?)- In the GH CI,
actions/checkout@v4will re-create an annotated tag as a lightweight tag during checkout, see also: #290 So without fixups (likegit fetch -f --tags) you cannot rely on tags staying lightweigth / annotated in some environments.
The distinction is not as clear cut in all implementations.
Hm, then I would say those implementations should be fixed. It should be Git which defines the meaning and Git documentation clearly states
Annotated tags are meant for release while lightweight tags are meant for private or temporary object labels.
The distinction is not as clear cut in all implementations.
Hm, then I would say those implementations should be fixed. It should be Git which defines the meaning and Git documentation clearly states
Annotated tags are meant for release while lightweight tags are meant for private or temporary object labels.
Interesting. I think that's new since I last used Git (in around 2015). The tag support we have right now is more similar to lightweight tags - they're just names, no metadata. I was hoping that would be sufficient. As usual when adding a feature, I think we should not worry about Git compatibility to start with. So I wonder what we need from tags if we ignore that. Perhaps this is a discussion for a different issue.
It should be Git which defines the meaning and Git documentation clearly states
Annotated tags are meant for release while lightweight tags are meant for private or temporary object labels.
For reference, the documentation note was only added in 2013: https://github.com/git/git/commit/29d55538b78fa8d916b99aef9a4b1e376df2dac0.
It seems likely that not all projects will adhere strictly to this guidance.
Interesting. I think that's new since I last used Git (in around 2015).
It's not really new.
It was always like that since I remember.
And the explicit explanation in the documentation was added in 1.8.4 back in 2013: https://github.com/git/git/commit/29d55538b78fa8d916b99aef9a4b1e376df2dac0
That for example describe does not show them is at least since 2007: https://github.com/git/git/commit/8713ab307940c37906631efb8ef96be37963f81c
The tag support we have right now is more similar to lightweight tags - they're just names, no metadata. I was hoping that would be sufficient. As usual when adding a feature, I think we should not worry about Git compatibility to start with. So I wonder what we need from tags if we ignore that. Perhaps this is a discussion for a different issue.
Well, yeah, how JJ implements and treats Tags might differ. Currently you cannot even create a tag as far as I'm aware. But when importing tags from Git, especially when collaborating with others using Git or when using a colocated Git repository which is the default now, I'd say that the Git meaning of tags should be respected in some way.
Also when JJ gets the ability to create tags, it should be able to create an annotated Git tag or always create an annotated Git tag most probably.
That a commit becomes immutable just because a lightweight Tag in Git was added feels quite wrong to me and just bit me, suddenly getting a new change created just because the lightweight tag appeared.
It seems likely that not all projects will adhere strictly to this guidance.
Imho people abusing lightweight tags should even be less a reason to derive any JJ behavior or meaning than what Git does. 🤷♂️
Many people use annotated git tags for storing release notes, when we want to keep our repositories as independent of GitHub as possible. See for example https://git.sr.ht/~mcepl/m2crypto/refs/0.46.0
Currently you cannot even create a tag as far as I'm aware.
You will be able to from the next release. There's a new jj tag set command that lets you create jj tags, which, as I said, are similar to lightweight tags. When exported to Git (which happens automatically in colocated workspaces), they become lightweight tags.
Also when JJ gets the ability to create tags, it should be able to create an annotated Git tag or always create an annotated Git tag most probably.
We have talked about adding a jj git tag command for creating Git annotated tags.
Nice, but still, if JJ tags are conceptually lightweight Git tags, they should not make commits immutable and probably should not be pushed by default, or JJ tags exported to Git should create annotated tags and only annotated tags imported as JJ tags.
With the current tag handling, JJ - Git interop - which I found quite great up to now - gets a big damper from my PoV, because if you use Git the intended and documented way, you get into trouble with JJ. :-/
if JJ tags are conceptually lightweight Git tags,
Ah, sorry, I didn't mean that they're conceptually similar. I just meant that they're technically similar. I think they are conceptually more similar to annotated tags now that I've learned that lightweight tags are supposed to be temporary.
I see.
Maybe then as long as tags in JJ do not have the additional metadata like a message, tags exported by JJ to Git should be annotated tags with some default message or maybe configurable by some Git-specific config which could be some template, and only annotated tags should be imported from Git, so that lightweight tags do not make commits immutable.
There could also be a git-specific setting to control whether lightweight tags should also be annotated as JJ tags for those projects that abuse lightweight tags, but the default should be to only import annotated tags.
If a specific message is needed in the annotated tag created by JJ like having the changelog in there as @mcepl showed,
this could then be some jj git tag that accepts a message as long as JJ does not model a tag with a message.
I'm not sure about conceptual differences or intent, but an important technical difference is that you can GPG-sign annotated tags, while non-annotated tags you cannot.
You will be able to from the next release. There's a new
jj tag setcommand that lets you create jj tags, which, as I said, are similar to lightweight tags. When exported to Git (which happens automatically in colocated workspaces), they become lightweight tags.
Are these jj tags to be intended to be used for release versioning?
At least for my projects, that's my main use of tags, in combination with git describe (which doesn't have a good jj equivalent so far).
Unless the jj tags are not meant for releases, shouldn't they be exported as annotated tags (e.g. by using the tags name as annotation or accepting a message argument), so they can be signed and used for git describe ?
Are these jj tags to be intended to be used for release versioning?
I would say yes, but keep in mind that I didn't know until today that lightweight Git tags are meant to be temporary. @yuja, what do you think?
I didn't know that lightweight tags are documented as temporary stuff either. I considered that annotated vs lightweight are just storage format differences, and tags in VCS are for release versioning, etc. in general.
We use lightweight tags as release identifiers, also many people seem to do this in public repos. In our case, we leave the release notes and change log in .md files. We don't use annotated tags--it's not worth the hassle to have release notes in both places, and we need people to have that info without access to the repo.
Sounds like whether to treat changes with what kind of tags as immutable should be configurable, and probably with patterns to allow naming conventions (e.g. we uses semver vs. date-stamp release identifiers to differentiate between production release candidates headed to formal verification and internal-use-only/unverified builds, for example).
It Would Also Be Nice (tm) to have jj be configurable to default to lightweight/annotated/auto-signed tags when adding a tag, too. Like jj tag set would be configurable to default to some combination of jj tag set --annotated or jj tag set --lightweight plus --signed or --unsigned.
I'd happily have my jj config default to --annotated and maybe even --signed if it were that easy, and added no operator friction.
It Would Also Be Nice (tm) to have
jjbe configurable to default to lightweight/annotated/auto-signed tags when adding a tag, too. Likejj tag setwould be configurable to default to some combination ofjj tag set --annotatedorjj tag set --lightweightplus--signedor--unsigned.
I still think that this functionality should be part of the jj git tag command discussed in #5426 since jj tag should stay backend-independent.
I still think that this functionality should be part of the
jj git tagcommand discussed in #5426 sincejj tagshould stay backend-independent.
Tags created by jj tag are currently exported to Git as lightweight tags. I think the suggestion was to have it configurable which kind of tags they should be exported as. I think that should work but there are probably some non-obvious decisions to be made (what all the metadata should be set to, how to handle config changes).
It is not only about exporting tags, but also about importing tags. When I created this issue no tag exporting even existed and in the released version it still does not.
The suggestion (my suggestion) is by default following the official backend logic. So as jj tags are meant as release identifiers, that would be only importing annotated tags from Git as jj tags and exporting jj tags as annotated Git tags by default. The message for the tag could something hard-coded or probably better some template that gets information like the tag name as keywords.
And if it is desired to support the misuse (according to Git terminology) of lightweight tags that some projects follow - no matter why they do it - then have one or two backend-specific settings that allow to configure which Git tags get imported as jj tags and how jj tags are exported to Git.
An explicit "make an annotated tag" independent of import / export settings would be more in the scope of a jj git tag command, yes.
Config change should probably handled like in other cases, so only newly found tags are imported according to the import settings and only newly exported tags are exported according to the export setting.
Addendum: If a tag is skipped to import due to that setting, a warning for that tag could be issued, so that you are hinted that you might want to change the setting if tags are skipped you want to have in jj.
And the import settings could additionally also support a string pattern, so that you can say "import all tags if they are matching pattern X" for those that use lightweight tags for both purposes but follow a naming convention.
Another consideration about the heavyweight tags is that they could be GPG signed, which many use as part of their securing repositiry (see that tag I linked above).
I think the suggestion was to have it configurable which kind of tags they should be exported as. I think that should work but there are probably some non-obvious decisions to be made (what all the metadata should be set to, how to handle config changes).
I mean the decision can only be made as soon as we define what the backend-independent jj tag creates, but it definitely seems wrong to make it configurable without knowing how it looks like (at the moment their just exported as lightweight tags, so we'll see). At worst this is defined by a explicit jj git tag import command or done via the git import/export mechanism for colocated workspaces.
The suggestion (my suggestion) is by default following the official backend logic. So as jj tags are meant as release identifiers, that would be only importing annotated tags from Git as jj tags and exporting jj tags as annotated Git tags by default. The message for the tag could something hard-coded or probably better some template that gets information like the tag name as keywords.
SGTM
An explicit "make an annotated tag" independent of import / export settings would be more in the scope of a
jj git tagcommand, yes.
I don't see a setting in scope for this at all, since there hopefully will only be one type of tag to export.
And the import settings could additionally also support a string pattern, so that you can say "import all tags if they are matching pattern X" for those that use lightweight tags for both purposes but follow a naming convention.
That rather sounds like a specific command for this use-case, not something a setting should do for you.
Is that really common in Git community to treat lightweight and annotated tags very different things? Since we didn't hear any complaints about this behavior until recently, I suspect many people wouldn't care much about their differences and just use them as "tags" transparently.
Even if we respect the documented use cases, it's up to user whether revisions marked by "private or temporary object labels" should be considered immutable or not. Since git tag doesn't allow to move lightweight tags without --force, lightweight tags can also be considered somewhat immutable.
Is that really common in Git community to treat lightweight and annotated tags very different things?
I of course cannot speak for the Git community as a whole. But I certainly hope so, and everytime I recognize a repository that uses lightweight tags for releases, I also tell them as often it is indeed due to simply not knowing better. But from my experience, yes they do. Lightweight Git tags are conceptually more like jj bookmarks (that they are exported to and imported from Git branches put aside). Annotated Git tags are conceptually like jj tags as we learned above.
When abusing lightweight Git tags for releases,
- you cannot have a description,
- you cannot sign it,
- you don't know who created the tag
- you don't know when the tag was created
- commands that do the differenciation (unless reconfigured by parameters) like
git describewill not consider the tag.
At work here for example you cannot even push a lightweight tag as this is technically prevented, due to the intended nature of lightweight tags.
There are for sure also many repositories using lightweight tags for releases, probably mostly simply due to not knowing better, hence my suggestion to make it configurable at least which tags are imported as jj tags (annotated / annotated and lightweight / following a naming convention). The exporting could just always do proper annotated tags, unless people really want to have them exported as lightweight tags or to provide symmetry.
The problem I see and experience is, that if you do use Git tags as they are intended, then it disturbs proper usage of jj, as I just marked @- with a lightweight tag for easy referencing by name for a short period of time in Git commands and suddenly @- became immutable just because jj treats them not like Git intends them to be treated.
This suddenly almost emptied the default jj log output when before I had like 20 changes there,
just because most of the chagnes were in the ancestry of the tagged commit.
Since we didn't hear any complaints about this behavior until recently, I suspect many people wouldn't care much about their differences and just use them as "tags" transparently.
That's quite possible. It can also be that many people just don't use lightweight tags at all, but only tag their releases and for that use annotated tags anyway as intended. Those will not have any problem so far as they simply do not have any lightweight tags that are mistreated. Those that do abuse the lightweight tags for releases also will not complain currently as jj behaves matching to their abusal.
Once the creation of tags in jj and exporting of those tags as lightweight tags in Git goes live, things might change. As then the people expecting annotated tags for releases might start to complain more. Besides that exporting jj tags that are meant for releases as lightweight Git tags which are not, of course makes the global situation worse, supporting and promoting that misuse. :-(
Even if we respect the documented use cases, it's up to user whether revisions marked by "private or temporary object labels" should be considered immutable or not.
If it depends on the user, then please make it configurable, but by default jj should imho follow the semantics defined by the backend.
Since git tag doesn't allow to move lightweight tags without --force, lightweight tags can also be considered somewhat immutable.
Yes, but that is two different things to talk about. The tag can be considered somewhat immutable. But the commit the tag points to is not. For a release tag, and thus canonically for an annotated tag, yes, the commit should be considered immutable by default. For a lightweight tag that just is a post-it sticker on a given commit for some time to find and reference it and that is never going to be pushed to any remote repository, it should not.
Is that really common in Git community to treat lightweight and annotated tags very different things?
For my part, yes. Lightweight tags are basically a strictly-worse kind of branch with a barely-overlapping less complete CLI interface. They are also not namespaced by remote so it's very easy to clobber tags on different remotes. They cannot be GPG signed. They are only "used" by accident when you forget to add -a to git tag.
I think Vampire's "abuse" language is overly forceful but TBH I have no idea what the intended use of lightweight tags is supposed to be. It feels to me like a deadend feature from the very early days of git that had to be kept around for historical reasons.
Annotated tags meanwhile are real git objects that you can attach metadata and signatures to. (Though they still have no namespacing.)
Since we didn't hear any complaints about this behavior until recently
AFAIK jj has basically no support for git tags and the standard advice is to use git tag to manipulate them. This issue is the first time I've seen active discussion about the issue.
I think Vampire's "abuse" language is overly forceful but
I'm sorry, I did not want to turn down anyone or anyone's usage with using "abuse", just to bring over the point that this usage is not how Git defines their intended usage. So we can also call it "non-canonical use" or "non-standard use" or any other term that transports that Git considers them different than they are sometimes used.
TBH I have no idea what the intended use of lightweight tags is supposed to be.
It is documented at the git-tag documentation as mentioned above.
AFAIK jj has basically no support for git tags and the standard advice is to use git tag to manipulate them. This issue is the first time I've seen active discussion about the issue.
That's not fully correct. As mentioned above, jj does import Git tags, making all Git-tagged objects immutable as they are imported as jj tags which are meant for releases, and in the trunk there is now also the possibility to create tags in jj that are currently exported to Git as lightweight tags, which I did not even know when creating this issue.
My understanding is that you can't create a github release with binary artifacts and (github UI) release notes without using annotated tags. I just recently started doing releases for the first time and got tripped up trying to use a lightweight tag.
I don't think it makes sense for jj tag to support both, given the overlap in functionality. It kind of seems to me like git lightweight tags are a lot like jj bookmarks. Maybe I'm missing something. I agree that jj shouldn't be beholden to the the backend data model.
I would prefer for jj tag to use annotated tags instead, given that that's the one you use for GitHub releases. Failing that, having jj git tag with support for both would be nice. I'm not terribly inconvenienced by having to fall back to git tag -a, but it is one thing that gets in the way of just uninstalling git outright. Idk how realistic that is in the medium term anyway.
Is that really common in Git community to treat lightweight and annotated tags very different things? Since we didn't hear any complaints about this behavior until recently, I suspect many people wouldn't care much about their differences and just use them as "tags" transparently.
Aside from signatures, there is a major difference for projects that use Git as the source of truth for versioning:
- The command
git describewill normally only pick up annotated tags to describe a release version. - The Github "Source code (tar.gz)" and "Source code (zip)" downloads have no
.gitdirectory, but version information can be embedded (throughgit archive) by using a$Format: %(describe:match=v[0-9]*.[0-9]*.[0-9]*) $string. See e.g. $Format$ example in jj-fzf, enabled by a corresponding .gitattributes file. - Like
git describe,$Format: %(describe)$will only pick up annotated tags. Unlikegit describe, there is no extra option to make it work with lightweight tags.
So in a nutshell, projects that seriously use Git for versioning MUST use annotated tags, since leightweight tags don't work with describe and signatures.
jj does import Git tags, making all Git-tagged objects immutable as they are imported as jj tags which are meant for releases,
Regarding immutability of commit objects pointed to by lightweight/annotated tags, I don't think it's wrong to make them immutable in jj. If a tagged commit was rewritten by git, the tag would remain at the old commit and new rewritten commit would be created. To mirror this behavior in jj, the old commit should be immutable, and the user would have to do jj duplicate and rewrite the duplicated commit.
In jj, bookmarks are used as named pointers to mutable commits, and tags could be used as immovable pointers? I said before that tags are meant for releases, but I didn't mean they shouldn't be used for other purposes such as tagging nightly versions, which Tim mentioned in https://github.com/jj-vcs/jj/issues/7908#issuecomment-3468085828.
In any case, I'm not against adding some config knob to export jj tags as Git annotated tag objects. Since jj wouldn't internally support tag objects, the metadata written to the exported Git tag objects would have to be configured statically, though.
I'm also not against adding some option to not import lightweight Git tags or some name patterns to specify importable tags, but I would say the default should be "import both lightweight/annotated tags". I checked random Git repositories I have locally, and some of them (including jj itself) use lightweight tags for versioning.
some of them (including jj itself) use lightweight tags for versioning.
Yeah, Martin already said that he simply was not aware that lightweight tags are not intended for releases.
I don't think it's wrong to make them immutable in jj.
Well, it was at least very unexpected to me that on adding the lightweight tag the jj log became almost empty,
and on deleting the lightweight tag I suddenly lost all those commits.
jj git init
jj commit -m a
jj commit -m b
jj commit -m c
jj commit -m d
jj log
# root(), a, b, c, d, and new change shown
git tag <commit id of c>
jj log
# root(), c, d, and new change shown (btw. why root() here?)
git tag -d foo
jj log
# a, b, and c automatically abandoned
# only root(), d, and new change left
for other purposes such as tagging nightly versions
To me a nightly version or milestone version, or release candidate version are all also kinds of release. Permanently labeling an unmodifiable state, opposed to having a temporary private label for something. If it is possible technically, an imho valid option would also be to treat all tags coming from or being sent to (so "existing on") a remote as immutable jj tags, but treating lightweight tags that are only available on the local Git repository differently or ignoring them unless they get pushed to a remote.
the metadata written to the exported Git tag objects would have to be configured statically, though
Which metadata would that be? Looking at https://git-scm.com/docs/git-fast-import#_tag again, there is name, target, tagger, tagtimestamp, and message. The name would be the name of the tag. The target would be the tagged commit. The tagger would be the user exporting the tag to the Git repo. The tagtimestamp would be the time the tag is exported to the Git repo. The message is probably the only thing questionable, and that could be statically configured, or as I suggested using some template that for example gets the tag name or maybe also some other information as keywords.
but I would say the default should be "import both lightweight/annotated tags"
The defaults are of course up to the discretion of you maintainers to decide, as long as it will be configurable, I would be happy.
I just think that by default import and export should adhere to the original intent of the underlying store, so that people using it as intended get the expected result by default, with the possibility to reconfigure it for the users that do not use the underlying store canonically. But that's just my opinion and 2ct of course. :-)