Support multiple Gradle builds in one repository (monorepo)

[sradi]

Hi,

in my use case, I would like to use axion release plugin for several independent gradle builds in the same repository. On Jenkins I am using a composite build, that kicks off all the separate builds in the same pipeline. Currently, I configured different prefixes to have distinct version tags for each project.

The problem is, that a new commit with changes in one of the projects also results in all other projects being released with an incremented version.

I am already working on a pull request for 'monorepo support' in axion release plugin, which would basically replace the condition isHead in VersionResolver/VersionSorter to find out, if the project is on a release tag. Instead, I would check, if there is a release tag on the latest commit, which contained changes in the rootDir of the current Gradle project.

@adamdubiel Can you imagine to include this feature, if I provide a pull request for it? As I am not very familiar with the codebase, I would appreciate a thorough code review.

[adamdubiel]

Yes, we could include this feature. Couple of things that you might want to watch out for:

• with monorepo, performance might be degraded as commit history grows - the less frequently updated dir is, the more commits you have to dig out from the history to find last tag
• it would be great t split the change into two PRs - first one introducing an interface to replace isHead with (with single implementation - checking for isHead :))
• I would appreciate if all new code was written in Java; I know that codebase around is mostly Groovy, but some of it is Java and it would be perfect if all new code was Java :) (in src/main, tests should stay groovy/spock)

[john-tipper]

@sradi, please may I ask how your monorepo is set up? Is it a Gradle multi-module build, where each module has a configuration section like the below, or are you doing something different?:

scmVersion {
    tag {
        prefix = '<prefix>' // where <prefix> is a prefix which distinguish tags that apply to that project only
    }
}

Please can you explain how the release process works for the different projects within the monorepo?

/top-project
    build.gradle // top-level build.gradle
    settings.gradle // include project1 & project2
    /project1
        build.gradle // defines prefix for project 1
        // other project1 files here
    /project2
        build.gradle // defines prefix for project 2
        // other project2 files here

I don't fully understand what versions should be returned for changes to files located within a sub-project directory vs changes made to files in both directories vs a change made to the top-level project?

[sradi]

Hi @john-tipper. Yes, I have configured a unique tag prefix for each build. But the repository does not contain one Gradle multi-module build, but multiple separate Gradle builds (whether multi-module or not), which are combined as a composite build on root level.

My goal was, to have one job in ci, which executes the composite build, but to prevent every nested Gradle build from getting an incremented version, even if only code in some builds has changed.

Actually, I was distracted from finishing this code change. I should soon get into this again, and provide clean solution for this.

[john-tipper]

Hi @sradi, thanks for the info, that's very helpful. I've been working on your initial PR so I'll push some revised changes soon for you and @adamdubiel to take a look at.

If changes are made to the base project (your composite project) then I think the base version should change, but as you say the nested projects should not necessarily have their version incremented. If a change is made to a child project then the parent project version also changes at present, which we don't necessarily want. If a child project (e.g. my project1 in the example above) has its version changed then none of its sibling projects (e.g. project2) should have their version changed. This last requirement is fixed with your PR, but the first two are not.

We basically want to be able to exclude paths from being part of the calculation to determine whether a commit is relevant to a version change. This is supported with an :exclude pathspec in the git log command, but not in JGit, see this SO question for details: https://stackoverflow.com/questions/59143934/java-jgit-how-to-get-git-commits-not-affecting-certain-directories.

I have submitted a PR to JGit to add the feature of excluding directories from the log() command, see https://git.eclipse.org/r/#/c/153673/ for progress of the PR. When that is released, we should be able to bump the version of JGit in axion (will also have to bump the grgit version to support v5 of JGit) and then use excludePath() when calculating the last relevant commit to exclude child folders, similar to how you call addPath() in your PR: https://github.com/sradi/axion-release-plugin/blob/monorepo-support/src/main/groovy/pl/allegro/tech/build/axion/release/infrastructure/git/GitRepository.groovy#L186

[john-tipper]

Also, as part of this change, I think we should remove the restriction that prefixes cannot include a subset of another prefix. It is not uncommon for child modules to share some initial characters and it would be great to be able to support a tag that is the module's name. E.g. for my example, the tags in the repo might be project1-, project2- and perhaps release- for versioning of the base project. I'll include this functionality in the PR.

[adamdubiel]

1. axion no longer. depends on GrGit :)
2. would be great to have separate PR which would focus only on removing the prefix restriction

[john-tipper]

Hi @adamdubiel

1. I'm not sure on this, I can see reference to org.ajoberstar.grgit.Grgit, amongst others?
2. Sure, PR #307 submitted. I can see a Travis failure on installation of JDK 8, but I don't think how that can be me that broke it!

[john-tipper]

@sradi, @adamdubiel I've updated Stefan's code to add tests and support to optionally allow the parent project version not to increment when a submodule's version is changed. Details are in PR #310.

[Dujma]

I see this one is still opened, but in the documentation there are mentions of monorepo approach. Is this now supported?

We have an use case where we have multiple gradle projects in a single repo and we are planning on using Axion for version management.

And in order to establish this process, do we need to have relationship between projects in gradle (like parent gradle project with childs) or we can isolate them completely in different directories?