Hotfixes - Support Patches for old minors and patches

[hipstersmoothie]

Is your feature request related to a problem? Please describe.

I want to be able to release a patch for an old minor

Describe the solution you'd like

1. If we merge into a tag we should be able to detect this and make a patch/minor release
2. these release numbers should use the build part of semver for minor/patch
3. Major version tags can version as normal, as there would never be a version conflict

[bnigh]

@hipstersmoothie Do you have any details of how this would be implemented / how it would look in practice?

Also, I'm not sure I'm understanding what merge into a tag means... could you potentially clarify? My understanding was that you could only create a PR with a branch as a target.

[hipstersmoothie]

No, I haven't really though about the problem too much.

Also, I'm not sure I'm understanding what merge into a tag means

The ui led me to believe that you might be able to merge into a tag, but upon further review this doesn't seem to be possible. If we could though this would make this possible.

Remembering a little though, I think this is what I was thinking:

1. Tag 1.2.3 exists, current version is 2.0
2. PR is made into tag 1.2.3, produces 1.2.3-0 (the 0 is the build part of semver)

[bnigh]

ok, utilizing build part of semver definitely makes sense. 👍

If it is not possible to make a PR directly to a tag, then this flow may require repo owners to create a branch from a tag on the upstream branch (target branch for PR). It would be slightly annoying, but I'm not sure if there would be a better alternative. Additionally, these types of releases are likely not common, so maybe this would be ok.

A similar approach to old major support (versionBranches) could be utilized where a branch prefix could be specified in the config. It seems like this feature may be targeted more towards hotfix type releases, so maybe makes sense to keep the configuration separate from old major support. What do you think?

---

An example flow may look like:

• user specifies in auto config to build using build identifiers for branches with prefix of hotfix-
• existing commit log

  * (tag: v2.0.0) (master)
  |
  * (tag: v1.2.0)
  |
  * (tag: v1.1.0)
  |
  * (tag: v1.0.0)
  

• repo owner creates a new branch for existing tag that needs hotfix

  * (tag: v2.0.0) (master)
  |
  * (tag: v1.2.0)
  |
  * (tag: v1.1.0) (hotfix-feature)
  |
  * (tag: v1.0.0)
  

• PR created against and merged to upstream/hotfix-feature (merge commit tagged with tag + build identifier)

  *  (tag: v2.0.0) (master)
  | 
  *  (tag: v1.2.0)
  |
  |  *  (tag: v1.1.0+1)
  | /
  *  (tag: v1.1.0) (hotfix-feature)
  |
  *  (tag: v1.0.0)
  

[hipstersmoothie]

A similar approach to old major support (versionBranches) could be utilized where a branch prefix could be specified in the config.

I like the idea but in practice I don't think it would be all that fun to use. Since auto releases a lot you would end up with a crazy number of branches.

If it is not possible to make a PR directly to a tag, then this flow may require repo owners to create a branch from a tag on the upstream branch (target branch for PR). It would be slightly annoying, but I'm not sure if there would be a better alternative. Additionally, these types of releases are likely not common, so maybe this would be ok.

I agree that this type of release is out of the norm. that makes automated branch creation less useful. You'd get lots of noise (more branches) without using them all that much, if ever.

---

As for your proposed flow above that seems to work well. It's unfortunate that we can't make PRs into tags though! would make this workflow way easier.

Thinking about this, this feature will probable need a few things into auto to be fully fleshed out:

1. New command auto hotfix: when run will release the project with a new hotfix version based on the latest tag in the branch
2. New functionality for auto shipit: Detect if we are in a hotfixBranch and run auto hotfix

As for how auto hotfix should work it could either go:

1. the way of next or canary => New hook that allows plugin to control how hotfix is released whether it's supported at all
2. re-use the version and publish hooks and make them able to release build semver

Of the two options I think I'm leaning towards 1 since calling the version and pubish hooks in a new way could be considered a breaking change. The drawback to that is some of the hooks wont be called afterVersion making some plugins not function. This is currently a problem for both canary and next though. as they don't call that hook either. So something that doesn't need to be worried about immediately.

[hipstersmoothie]

Are you interested in trying to add this?

[bnigh]

I like the idea but in practice I don't think it would be all that fun to use. Since auto releases a lot you would end up with a crazy number of branches.

oops, meant it was similar to old major support in terms of having a branch-prefix config. Definitely agree that automatically creating branches would create too much noise, especially with frequent releasing, 😆

Thinking about this, this feature will probable need a few things into auto to be fully fleshed out:

1. New command auto hotfix: when run will release the project with a new hotfix version based on the latest tag in the branch
2. New functionality for auto shipit: Detect if we are in a hotfixBranch and run auto hotfix

yep. I think that would largely encapsulate the changes required.

As for how auto hotfix should work it could either go:

1. the way of next or canary => New hook that allows plugin to control how hotfix is released whether it's supported at all
2. re-use the version and publish hooks and make them able to release build semver

Of the two options I think I'm leaning towards 1 since calling the version and pubish hooks in a new way could be considered a breaking change. The drawback to that is some of the hooks wont be called afterVersion making some plugins not function. This is currently a problem for both canary and next though. as they don't call that hook either. So something that doesn't need to be worried about immediately.

High level, I think 1 makes sense as well. I would need to trace through the code to have a better understanding of which hooks are called for which command, but if next, canary, & hotfix all follow similar patterns, then any pain points would likely be easier to solve later.

---

Are you interested in trying to add this?

Sure 👍, I'd be happy to. I probably won't be able to take a look at implementation for this until next week, but am guessing that is ok since this issue hasn't been updated since march.

[hipstersmoothie]

Awesome! No planned work on this one so it's all you

[bmuenzenmeyer]

FWIW - I was going to attempt to construct this as a plugin by doing exactly what you regarded as an inferior choice - making a version-MAJOR-MINOR branch. It doesn't feel like too much noise over our current process, TBH. We already create branches for each release line.

A major portion of versionBranches right now is this plugin:

    this.hooks.beforeCommitChangelog.tapPromise(
      "Old Version Branches",
      async ({ bump }) => {
        if (bump === SEMVER.major && this.config?.versionBranches) {
          const branch = `${this.config.versionBranches}${major(
            await this.hooks.getPreviousVersion.promise()
          )}`;

          await execPromise("git", [
            "branch",
            await this.git?.getLatestTagInBranch(),
          ]);
          await execPromise("git", ["push", this.remote, branch]);
        }
      }
    );

It looks trivial to add support for SEMVER.minor, but I'll admit I don't know what other parts of auto may need to change.

[hipstersmoothie]

That's part of the what happens for old release but then this check in shipit also has to pass

https://github.com/intuit/auto/blob/f37945b45b7fef6ffdb00ffa0250de449e02b883/packages/core/src/auto.ts#L1442

Which then goes to here and basically just overrides where to start calculating the release from

https://github.com/intuit/auto/blob/f37945b45b7fef6ffdb00ffa0250de449e02b883/packages/core/src/auto.ts#L1509

The one consideration we have to make with old minor/patch and not major is the potential version mismatch. With minors it is probably less of a problem though

*  (tag: v2.0.0) (master)
| 
*  (tag: v1.1.1)
|
|  *  (tag: v1.1.2) <- Need to add logic to find an available tag 
| /
*  (tag: v1.1.0) (hotfix-feature)
|
*  (tag: v1.0.0)

So maybe hotfix command isn't needed. Instead it could just try to unique version number. There should might need to be some validation around this though.

Rules:

1. can't release major on any old minor/patch release branch (required)
2. can't release minor on any old patch release branch (maybe unnecessary?)

[bnigh]

@hipstersmoothie Do you know what the default behavior is now if you have ci setup to build off of an older tag, where the next tag is not available?

My guess, is there would be a failure, although I'm not sure where this error would occur (maybe at tag push?).

ie:

*  (tag: v2.0.0) (master)
| 
*  (tag: v1.1.1)
|
|  *   <- what is current behavior if I try to release this commit
| /
*  (tag: v1.1.0) 
|
*  (tag: v1.0.0)

---

Outside of enforcing the rules @hipstersmoothie specified, I see one potential problem / confusion with the try to unique version number strategy / implementation. For the example @hipstersmoothie gave example, v1.1.2 would be seen by others as a patch release from v1.1.1, but since the development wasn't linear, this is not guaranteed. Potentially, there could even be a backwards incompatible change from v1.1.1 to v1.1.2 since it is likely that v1.1.2 doesn't have the changes from v1.1.1. This could become exacerbated and more confusing if the next unique version is a larger distance from v1.1.0 (ie: what if the next unique version is v1.1.100?)

Utilizing build metadata part of semver would limit this confusion since according to semver spec, it is ignored when calculating version precedence. If incrementing build metadata number, then perhaps commit sha could be used instead of an incrementing number.

In general, software development is not necessarily linear, so not sure there is a best implementation.

---

If we want to generalize, then perhaps we could have some sort of config or hook to specify the strategy for above regardless of branch (or branch could be a parameter to hook). This way, different implementations could be used via plugins.

[hipstersmoothie]

Do you know what the default behavior is now if you have ci setup to build off of an older tag, where the next tag is not available?

Usually tags don't build because the commit has [skip ci] in the message

[hipstersmoothie]

This could become exacerbated and more confusing if the next unique version is a larger distance from v1.1.0

This is a great point. Definitely makes the build metadata portion of the version the right strategy.

If we want to generalize, then perhaps we could have some sort of config or hook to specify the strategy for above regardless of branch (or branch could be a parameter to hook). This way, different implementations could be used via plugins.

your post pretty much convinced me that finding a unique tag is a confusing scheme to go off and we probably shouldn't.

[hipstersmoothie]

Maybe if we do a mix though it could work. Patching and old minor should be easy. It could work in the exact same way as oldVersionBranches for majors:

*  (tag: v2.0.0) (master)
| 
*  (tag: v1.2.0)
|
|  *  (tag: v1.1.5) <- Can merge patched into old minor branch, maybe error when PR has `minor`
| /
*  (tag: v1.1.4) <- branch: version-1.1
|
*  (tag: v1.0.0)

Then we only need to do the build metadata portion for patching patches.

[mattfelten]

With the from and useVerion options merged, is there a built-in way to do a hotfix? I just had to make one today and opted to do it painfully manually.

For context, my project is on 7.7. A major consumer of our project is currently releasing a version that is on 7.5, found an issue and required 7.5.1 so they didn't have to re-QA everything mid-release. Not optimal, but does occasionally happen.

[hipstersmoothie]

Still no way to do this to my knowledge without manual intervention. I think someone internally here at intuit plans on adding this eventually. I agree this is a sorely missing feature in auto

[mattfelten]

Awesome to hear! Thanks 🙏

[bmuenzenmeyer]

We (me and @10hendersonm) developed a solution for this but it's quite involved. It solely uses auto and plugins however!

We just hotfixed 7.2.1, 8.1.1, and 8.2.2 of our project using only PRs (very carefully).

We could look into how to share this out if people are interested?

On Thu, Jan 14, 2021, 7:27 PM Matt Felten [email protected] wrote:

Awesome to hear! Thanks 🙏

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/intuit/auto/issues/1054#issuecomment-760583830, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACI3Q6RKDONYJVH6UWUPFLSZ6KZNANCNFSM4LFJ4ULQ .

[hipstersmoothie]

I would be very interested!

[sumwatshade]

Hello all! I am interested in contributing this. My proposal is the following:

1. In general, auto will attempt to respect any version-* branches. For example a patch merge from any branch into version-1.1.4 will generate a release of version 1.1.5. If that version has already been created, then either the NPM release or the Git Tag will fail, but that a is normal and expected error case
2. We can update versionBranches to accept "major" | "minor" | "patch", which will generate potential hotfix branches based on what you specify. This allows you to choose the trade-off between manually needing to create a version-* branch and ending up with way too many branches to maintain when you don't need fixes

Thoughts?

[hipstersmoothie]

but that a is normal and expected error case\

I'd still expect this to create some type of version. I recently had a case where I wanted to release a hotfix from a specific commit as to not pull in anything else. I think in this case we could use the build part of semver from the conversation above.

We can update versionBranches to accept "major" | "minor" | "patch"

The current config can either be true or a string to prefix major branches with. I think the new config would have to look something like

{
  "versionBranches": {
      "branchPrefix": "version-",
      "types": ["major", "minor"] // defaults to just major   
   }
}

---

The last question to answer is still hotfix hook or call version+publish hooks looking at the code for the current old version branch implementation we do option 2 and call version+publish (and I think this mistakingly publishes to the latest tag).

@lshadler We many need to pair on this for a little while to find the best approach.

@bmuenzenmeyer Can you give an overview of your approach?

[hipstersmoothie]

The more I think about implementation the lot I think this will necessitate v11 and more context added to the version and publish commands.

For old releases we need the full latest pipeline to run with changelog, afterVersion, and all or their hooks called during a latest release.

We should probably make next releases have the same version afterVersion publish afterPublish flow latest does. That can be a part of v11 too. It should match the latest workflow so you can added similar automation.

Canary workflow can stay the same since nothing gets committed for a canary and you can already just tap the canary hook to do more things.

[bnigh]

Hello all, with the craziness of this last year, unfortunately this issue has gotten away from me.

Regarding the different approaches, I think it may help to consider the different use cases these features solve for. In my opinion, I can see 2 use cases:

• (1) long term support for an old branch (ie: old major version)
• (2) short term support for a specific version (ie: hotfix)

Based upon this conversation, I think it makes sense to have distinct approaches for each of those use cases.

(1) For long term support of a branch / release track, I believe versionBranches should be able to solve for. If there is a desire to extend this behavior for minor versions (as a few have have mentioned above), then that could be an enhancement to that functionality.

(2) For short term support / hotfix, based upon threads above, I think there should be standard way for users to always use build part of semver to generate a hotfix release. This gives consistent behavior for Code Owners to use and avoids a few complicated corner case scenarios. For this case, I think a new hotfix command and hook could be used. This could be a distinct enhancement. In general, this use case should be relatively rare as consumers should be encouraged to use the latest versioning if possible.

edit (For the short term use case, potentially versionBranches config could be extended to allow a parameter / toggle / flag that denotes whether to honor the semver labels for the version- branches or to always utilize the build part of semver and ignore any labels for those branches)

---

Are there any other use cases other than these 2 that should be considered?

Should different approaches be considered for different use cases?

(also, I can still help out with some of these changes, but definitely don't want to block anybody else from taking up changes for this)

[bnigh]

also, a tangential thought for branching patterns:

auto will generate a git tag (release) for releases. This means that a valid git ref is pushed to github. For the use case of short term support (ie: short lived branch / hotfix branch), this means a user can delete the short lived branch after a release / git tag has been pushed.

ie (click here for example scenario)

• given a master branch with following tags

  *  (tag: v1.3.0)  (master)
  | 
  *  (tag: v1.2.3)
  

• create a new short lived branch from a specific commit (ie: hotfix-1.2.3)

  *  (tag: v1.3.0)  (master)
  | 
  *  (tag: v1.2.3)  (hotfix-1.2.3)
  

• push changes / merge PRs into short lived branch

  *  (tag: v1.3.0)  (master)
  | 
  |  * (hotfix-1.2.3)
  | /
  *  (tag: v1.2.3) 
  

• which can generate a new release / tag upon PR merge

  *  (tag: v1.3.0)  (master)
  | 
  |  * (tag: v1.2.3+1)  (hotfix-1.2.3)
  | /
  *  (tag: v1.2.3) 
  

• since the new tag is a valid git ref tracked by github, this means code owners can delete the short lived branch and still have reference to the new commit / release via the tag (v1.2.3+1)

  *  (tag: v1.3.0)  (master)
  | 
  |  * (tag: v1.2.3+1)
  | /
  *  (tag: v1.2.3) 
  

For short lived branches, it may be good to document above branching pattern when adding feature. I've personally found that many are not aware that you can delete the branch and still have reference to the new commit, which may help different projects to reduce clutter of short lived branches.

[bnigh]

I was recently playing around with code to utilize build part of semver to create builds. As I was doing so, I came across the use case where the latest github release wasn't necessarily the latest/highest semantic version.

For instance, in the example mentioned here: https://github.com/intuit/auto/issues/1054#issuecomment-780286683, the latest github release would show as v1.2.3+1 since it was temporally created after the highest version semantic version: v1.3.0.

Since quite a few places in code reference getLatestRelease, this can lead to varied behaviors if the pipeline doesn't explicitly set the from parameter. ie:

• what is included in release notes
• what the previous version is calculated as
• potentially breaking functionality if latest github release is not reachable from HEAD commit

I haven't tested, but I believe these types of scenarios would also be reachable via existing versionBranches behavior. I believe this is related to the comment regarding which flows should generate git tags / github releases:

The last question to answer is still hotfix hook or call version+publish hooks looking at the code for the current old version branch implementation we do option 2 and call version+publish (and I think this mistakingly publishes to the latest tag).

---

In terms of resolving, this issue can likely be resolved independently of hook refactoring by replacing getLatestRelease logic with either:

• (1) fetch github releases using listReleases and then sort to find highest release version (no prerelease or build parts) that is reachable
• (2) or fetch git tags and then sort to find highest release version (no prerelease or build parts) that is reachable

The difference here would be whether auto views github releases or git tags as source of truth, which is related to discussion https://github.com/intuit/auto/discussions/1867#discussioncomment-684192.

I would initially lean towards (2), as

• (a) we are ultimately after the git ref (tag) and not the other elements of the github release
• (b) it would reduce the number of network calls

@hipstersmoothie, If we need to modify the getLatestRelease logic, what are your thoughts on (1) using github releases vs (2) using git tags vs (3) something else?

[hipstersmoothie]

Yeah for multi package auto to work it will need to refactor all the latest GitHub release stuff to just tags. 2 is def the option to go with to get this work done.

[tl-NASEEBULLAH-AHMADI]

Hi guys! Can I get an ETA on this, please? Is there a way to support patches for old minors yet or is there a workaround for it until auto implements this feature?

[cbackas]

We (me and @10hendersonm) developed a solution for this but it's quite involved. It solely uses auto and plugins however! We just hotfixed 7.2.1, 8.1.1, and 8.2.2 of our project using only PRs (very carefully). We could look into how to share this out if people are interested?

@bmuenzenmeyer @10hendersonm Did this solution end up being no good long term? Poked around your pages and dont see any repos for a plugin, did you decide to keep the plugin "internal"?

[bmuenzenmeyer]

Yes it's currently internal. Is there interest in seeing it?

[cbackas]

Yes it's currently internal. Is there interest in seeing it?

Yes I'd personally love to see what you did to solve this issue!

[10hendersonm]

We've got a handful of mini plugins we may be able to clean up into a single one once we re-grok how they're working. For now, a written description may have to do.

A couple caveats:

• This is just how we do it. It doesn't have to be this way, it's just what's worked for us.
• We're actually back on [email protected]. I hope the advice given remains true up to latest.

Workflow

• New features (major, minor) are PR'd at the baseBranch (or a prelreaseBranches)
• The baseBranch is not actually a representation of latest, the highest release-${major}.${minor}.x is the representation of latest.
• patches are always targeted at an "old" version branch (release-${major}.${minor}.x), whether or not they target the latest code in npm
• We have a chart we refer to as the "Pepe Silvia chart" which shows the process of patching a very old version and raising the change through to latest. I think it looks something like this:

  # INITIAL PATCH
  # check out oldest impacted version
  git checkout release-1.0.x
  # ensure up-to-date
  git pull
  # create feature
  git checkout -b my-patch-branch
  echo "Don't forget to enable CORS" >> README.md
  git commit -am "fix: Makes it a userland problem"
  git push
  # PR `my-patch-branch` to `release-1.0.x`, creating a `1.0.1` release
  # Make sure you get the updated post-published release-1.0.x branch:
  git checkout release-1.0.x
  git pull
  
  # RAISE PATCH UP
  git checkout release-1.1.x
  git pull
  git checkout -b rc-1.1.1
  git merge release-1.0.x
  # Good luck with the merge conflicts
  git push
  # PR `rc-1.1.1` to `release-1.1.x`, creating a `1.1.1` release
  git checkout release-1.1.x
  git pull
  
  # Raise further as necessary
  git checkout release-2.0.x
  git pull
  git checkout -b rc-2.0.1
  git merge release-1.1.x
  git push
  # PR `rc-2.0.1` to `release-2.0.x`, creating a `2.0.1` release
  git checkout release-2.0.x
  git pull
  
  # (Optional) Lift into a prereleaseBranches so it's included in the next `major` / `minor`
  git checkout beta
  git pull
  git checkout -b 1.1.1-prerelease-maintenance
  git merge release-2.0.x
  git push
  

Changes

.autorc.js Config

Mostly you just need versionBranches set. We use release-. You'll see why later.

Delete auto-generated versionBranches branches immediately

We have an afterRelease plugin that takes the lastRelease version and checks semver.major(lastRelease) to determine if a release-${major} branch exists. If it does, we delete it off the remote with git push origin --delete ${oldBranchName} (ex git push origin --delete release-1). I suppose if you wanted to preserve these you could add some logic to retain them as a stand-in for what would have been the release-${major}.0.x branch.

Push a new versionBranches that tracks the minor

auto doesn't validate the whole branch name for versionBranches, only that it starts with the given value. It doesn't care if you're pushing release-1 or release-1.0.x, it's going to walk through the old version publish flow anyway.

afterRelease, if the current branch is the baseBranch, push it onto the old version branch with git push origin "${baseBranch}":"${versionBranch}" (ex git push origin "main":"release-1.0.x").

(Optional) While you're pushing branches off of a freshly released baseBranch, we take advantage of this moment to push onto our preleaseBranches (beta) as well so it's automatically dragged up to date.

"Manually" tag up latest on patches

auto will only handle creating latest when publishing your baseBranch, but as stated in the Workflow above, patches don't land on your baseBranch any more, they'll target the highest old version branch.

If the current branch is an "old" version branch (release-${major}.${minor}.x), detect if the version that was just released is higher than the latest available version. If so, you'll have to create automation to dist-tag the package(s) to latest.

---

Notes:

• You might actually still be able to patch onto your baseBranch. We don't because we stage work on our beta prerelease branch throughout the month, so pushing onto our production base branch would bypass our normal workflow in a weird way. Instead, we consider the current release-#.#.x branch to be the definition of latest rather than the production base branch.
• I can't remember if this was related or not, but you might need to do something to retain skipRelease / internal functionality. I think that was mostly on us for sneaking commits in mid-publish, though.