Pin versions

[realdadfish]

My use case might sound a little bizarre at first, but hear me on. I have a Gitlab CI pipeline that operates in different stages. All these stages do not share the same workspace, but start with a blank one - and a freshly cloned Git repo - by default. (Gitlab does not support shared workspaces.)

My workflow nonetheless should / must look like the following:

• stage1: createRelease
• stage2..n-1: use release version in build
• stagen: pushRelease

As of now this is not possible, because I'd have not only to create the tag, but push it already in stage1 so it becomes available in stage2..n-1, but I'd also would have to remove the tag from the remote repo again in case stage2..n-1 fails. Finally, stagen becomes superfluous because the tag had to be pushed to the repo already anyways.

So, what I came up with was a custom Gradle Plugin that overrides / uses the Axion infrastructure to do the following:

• have a new pinVersion task that writes out the output of scmVersion.version to a configurable file
• transport this file through the different stages
• have a custom myPlugin.version method that either reads out the file (if it exists) or proxies to scmVersion.version and a custom OutputCurrentVersion task that does the same
• have a custom CreateRelease that does the same

Now while step 1 to 3 was kind of easy to implement, step 4 wasn't. The problem is that the VersionConfig instance has no interface it implements (so I could write an easy proxy implementation) and both cglib and javassist threw errors because they couldn't cope with the fact that the actual instances of the class was VersionConfig_Decorated (thanks Gradle) and not VersionConfig.

Anyways, I had to resort to the ugliest implementation, basically proxying all methods in VersionConfig myself and only overriding the needed VersionConfig.getVersion() method that is also used by the CreateReleaseTask.

Now this all works, but it is a great hack. I wonder if there would be the possibility to make this easier, either by implementing something like "pinned versions" directly into the plugin itself or at least by making it easier to override the contents of the VersionConfig structure for "bizarre" use cases like mine :)

What do you think?

[realdadfish]

@adamdubiel Any feedback on this here?

[bgalek]

@realdadfish I think interface extraction of VersionConfig would be ok - let's agree, your use-case is very specific ;)

[realdadfish]

@bgalek I think its not so uncommon with today's standard to have a multi-stage isolated pipeline, all cloud build providers isolate a certain stage from the next and unless you cache the whole git repository in between (which you usually don't do because caching many small files is slower than just cloning a repository again, maybe even utilizing shallow cloning).

So this plugin works still nicely for all kinds of pipelines that share a common workspace between the stages (like Non-Cloud Jenkins does), but actually this is no longer the norm.

[realdadfish]

I think interface extraction of VersionConfig would be ok

How would this look like exactly for you? VersionConfig is just a data holder / configurator for the plugin.

[bgalek]

@realdadfish as far as I know most popular CI/CD pipelines use branches/snapshot/candidates stage 1 - push release-1.0.1-SNAPSHOT (or i.e. release-1.0.1-mybranch-SNAPSHOT) stage 2 - test actual release-1.0.0 VS release-1.0.1-SNAPSHOT stage 3 - release release-1.0.1 to the public if got to stage 3

I would need to know a little more about your project to understand your needs better.

I understood, that if we could create you an easy way of overriding/composing VersionConfig.getVersion() you could clean up your proxy class.

Sorry if I don't get it correctly yet, please bear with me ;)

[realdadfish]

So, the general use case that I have in most projects is the following. I have three or more stages:

• The first stage creates a non-snapshot release version based on the current rules, i.e. 1.2.0-SNAPSHOT becomes 1.2.0
• The second or any subsequent stages needs this 1.2.0 version marker and creates artifacts from it (be it binaries that contain the version compiled in, changelog files that reference the version, you name it)
• The third or any last stage creates the actual tag for the version in the SCM and pushes it (after all other artifacts have been pushed)

Axion already supports this workflow by distinguishing between createRelease and pushRelease in two respective Gradle tasks. The first one adds a local git tag to fixate the version, the second then pushes the release to the remote. This works all fine when the repository between these two stages is shared, i.e. in a shared workspace like good old Jenkins supports it.

However, if the stages are itself isolated, the tag that is created in createRelease isn't automatically moved over to the stage where it is needed, nor should it via git push --tags! The existance of the tag would mean that the release has already happend and in case the subsequent build stages would fail, this tag would actually have to be removed / cleaned up by hand.

Thats why I came up with the idea of "version pinning". Instead of marking the release version in the Git repository, I write the needed information into a file, that can be easily moved over to subsequent stages. If Axion is then executed there, it finds the file and reads the release information from there instead of looking at the repository (which still doesn't know anything of the release tag).

[realdadfish]

@bgalek Whats your stance on the above?

[realdadfish]

Brought the implementation up-to-date with 1.14.3

[big-andy-coates]

@realdadfish could you not just run createRelease again at the start of all the intermediate steps to put the repo into the correct state? e.g.

• step1: clone repo, run createRelease to add local tag, do stuff.
• step2: clone repo, run createRelease to add local tag, do other stuff.
• step3: clone repo, run createRelease to add local tag, run publishRelease to push tags.

???

[realdadfish]

In theory I could do this, however it would not be safe. Since the repo is cloned anew each time, in case a new tag arrives on an earlier revision (or is removed), the outcome of createRelease changes. I really want to have one step where the version number is determined and then be absolutely sure that this does not change during the subsequent jobs.

And this is not only about a "bad timing" corner case, imagine a nightly job fails due to some temporary issue and you want to retry / kick-off the original pipeline hours later. The whole world could have been changed by then.

[realdadfish]

I'm closing this since it seems to be infeasible to be implemented here.