concourse
Git tag backend
We'd really like to use git tags as the backend for semver so that we can automatically version our code repositories. It's possible to implement this now using a separate semver storage repo and the git resource, however it's possible for the two repositories to fall out of sync.
Opening this here as we've expressed interest in working on it in Slack and were told it would be accepted upstream.
Hi there!
We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.
The current status is as follows:
- [ ] #112052101 Git tag backend
This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.
There's no update on this in Tracker. @Jonty, did you start work on it? I'd be quite interested in this, as the idea of having version numbers stored in a disjoint branch/bucket seems a bit wrong to me.
@DanielJonesEB No, we've never started work on this - it's still lingering somewhere in our backlog! We're exceptionally interested if anybody does though.
I'd really love for this to support Git tags rather than a file pushed to the Git repo. 👍 on this being a good idea
I've started work on git tag support, but there still is some testing and debugging to be done before it is PR worthy let say ;-)
But I'm running into some choices that are not entirely trivial, so I'd like to see if any one has some input on them:
- Git tags support messages, should I implement support for annotated tags? Or should I keep it compatible with the rest of the drivers?
- git describe format: git can output {last-tag}-{num-of-commit-since}-{commit hash}, the latter part could be used as a build number of sorts. Currently I drop it, but I could implement support for it.
- Pushing/rewriting tags: git doesn't allow duplicate tags, so to have similar behaviour as with the file implementation I have to force tagging and pushing tags. I don't think that that is a great idea; but without it the semantics of the resource kinda change.
@holstvoogd Awesome!
- Maybe the annotation could just be information about the build that produced the version, and when it was created? I wouldn't be too keen to expose it through params as it'd make it different from other backends.
- Usually the version will be propagating through the pipeline alongside the commit (git resource) that it pertains to, so I wouldn't worry too much about providing extra data. RC versions are also already automated through the current interface.
- It should definitely be idempotent. But we definitely shouldn't do a force-push. I'm not seeing anything that's atomic, but a simple
git push origin :(tag)before creating the tag might be the safest option. :/ A similar issue came up in the Git resource: https://github.com/concourse/git-resource/issues/72
You should absolutely be using annotated tags, as the commits will be in a proper repository and you want the time/date/author information - lightweight tags are really only for quick local use.
quick update on where I'm at:
- tags are now annotated & get the pipeline, job and build name in the message.
- the whole test suite for git is now also run against the git-tag driver.
- still working on extracting the 'best' tag for a commit from git. git describe seems somewhat unpredictable in its behaviour if there are multiple tags on one commit. (As in: on my mac it returns the most recent, on the buildroot container it returns the oldest tag... Even using the same version of git >_<)
As for making this atomic & idempotent: It seems the best way to keep the behaviour equal to the file base backend is to always use force on creating/updating the tag AND also force push the tag specifically. The only way to 'update' a tag is to force push it or to delete and recreate it. So this is basically the same as preforming a delete & create both locally and remote; but a bit more atomic.
Alternatively we can just error if the tag already exists and fail the build at that point I guess. But that breaks the test-suite in place for the git backend.
I think delete and recreate makes more sense. Force pushing is often disabled, e.g. that's a new GitHub feature (and for us somehow got enabled for every branch). It also sounds scary.
On Wed, Aug 10, 2016, 8:28 AM Arthur [email protected] wrote:
quick update on where I'm at:
- tags are now annotated & get the pipeline, job and build name in the message.
- the whole test suite for git is now also run against the git-tag driver.
- still working on extracting the 'best' tag for a commit from git. git describe seems somewhat unpredictable in its behaviour if there are multiple tags on one commit. (As in: on my mac it returns the most recent, on the buildroot container it returns the oldest tag... Even using the same version of git >_<)
As for making this atomic & idempotent: It seems the best way to keep the behaviour equal to the file base backend is to always use force on creating/updating the tag AND also force push the tag specifically. The only way to 'update' a tag is to force push it or to delete and recreate it. So this is basically the same as preforming a delete & create both locally and remote; but a bit more atomic.
Alternatively we can just error if the tag already exists and fail the build at that point I guess. But that breaks the test-suite in place for the git backend.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/concourse/semver-resource/issues/5#issuecomment-238903825, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAHWCtMBG4qy9uUtixRL_N3wHJDXNuLks5qee4bgaJpZM4HInOP .
Not sure how much this fits in with other version libs like: https://github.com/nebula-plugins/gradle-git-scm-plugin
Any thoughts?
Agreed, I've implemented it as a two step process now & fully tested it. I'll make a PR of it in a sec.
On a side note: I'm pretty sure force pushing a tag is not protected since tags are outside of the whole branch system.
@darrenhaken I looked at that lib, they actually don't allow you to update or delete tags. They push tags with a git push --tags which is considered a bad practice because:
I recommend not using or training others to use git push --tags as it can be very very difficult to get rid of bad tags when your co-workers are trained to push all tags, as people continue to push the old bad tags they have locally every time they want to push a new tag. Because of this, I will only every advise someone to use git push origin <tag_name> now. – Scott Jungwirth
@holstvoogd thanks for clearing that up.
I'll have to read more about good practices for tagging.
Excited for this to come out 👍 keep up the awesome work
sorry, i've been on vacation for a few weeks :)
I've made some progress, but also discovered this approach won't work for my companies needs. So I can't give it as much priority as I'd like right now. But I hope to make some progress this week.
It seems that @holstvoogd isn't able to work on this anymore. However, it looks like his work can be found at https://github.com/holstvoogd/semver-resource. Perhaps someone else could help get it over the finish line?
@erickt @willejs Yeah, sorry, I can't finish this. I have no way of testing/using it at the moment and haven't even looked at go code in over a year ;) But it should not be super complicated to finish this if I recall correctly. Most of the plumbing should be good, but there are some issues because I tried to make this do to much that was not necessary. There should be an old PR of this with some discussion on what needed to be done.
Bumping this because it would be really nice to have. Is it still feasible to get @holstvoogd 's code finished and merged?
