🔧 Repository: maintaining multiple release lines of plugins

[BethGriggs]

Some plugin owners may want to maintain and develop multiple versions of their plugin in parallel.

My initial implementation idea:

• For the few cases where this is required, the plugin owner can create a branch named plugin/vN.x where N is the major version line they wish to maintain in parallel.
• We have a generic and manually triggerable workflow, which reuses the changesets automation, where we can supply both the branch and workspace to be released.
• Only that plugin/workspace gets released from that branch.

[BethGriggs]

An alternative workaround might be to allow plugin maintainers to add multiple directories. That way no extra branches or new workflows should be required - they just need to create a changeset in the specific plugin directory.

[BethGriggs]

The previous suggestion in https://github.com/backstage/community-plugins/issues/860#issuecomment-2293528905 will not work as yarn workspaces detects the duplicate name and errors, so the branch option looks to be reasonable.

I figure these could even be short-lived branches when a release is needed, as we have the tags as a permanent reference.

[nickboldt]

short-lived branches when a release is needed

So your suggestion is that we 'd have a branch called workspacename/x.y.z for each maintenance build?

eg., argocd/1.24.7 to release the 1.24.7 version of the argocd plugin (to support an older Backstage version), while main branch is at version 1.25.1 (aligned to the latest Backstage version)

and if we needed another bugfix we'd have a new argocd/1.24.8 branch... rather than reusing a argocd/1.24 branch for all the updates?

[BethGriggs]

and if we needed another bugfix we'd have a new argocd/1.24.8 branch... rather than reusing a argocd/1.24 branch for all the updates?

I figure the workspacename/x.y.z branch doesn't need to always need to be an active working branch and therefore doesn't have to permanently exist between bugfix releases to be able to ship a bugfix/critical patch:

Something akin to:

git checkout tags/v1.24.7 -b argocd/1.24.8
git commit | | git cherry-pick 
git push origin argocd/1.24.8

# Run release workflow actions which automatically pushes the new tag

git branch --delete argocd/1.24.8

(Repeat the above if a second bugfix is needed at a later date)

[Rugvip]

Could you outline some of the needs that lead to this requirement? In the main backstage/backstage repo out only use of multi-track releases has been critical security fixes. Is it the same here and is that the only thing we should solve for? The main requirement for that is to have a publishing set up for a separate branch pattern, and also have strict branch protection rules for that pattern.

We wouldn't want to copy out patch process straight from backstage/backstage though. Without a fixed release cadence you don't end up with the same complexity of needing to leave space for previous patch releases that we have in the main repo. For example, if 1.3.0 of a package goes out in main-line, then 1.3.1 in a patch, we have logic to make sure that the next main-line release skips straight to 1.3.2.

[BethGriggs]

In the main backstage/backstage repo out only use of multi-track releases has been critical security fixes. Is it the same here and is that the only thing we should solve for?

Critical security fixes are a core reason, but they're not the only aspect this is useful for.

I believe some of the Janus plugins have required breaking changes when updating to support a new Backstage version. Plugin consumers may not always be in a position to update their Backstage version, so we may still need to deliver fixes (likely only critical/important) to the version of the plugin that is compatible with their earlier Backstage version.

Other reasons may be tied to the service the plugin provides - a plugin that integrates with a service that has a new major versions with different APIs may prefer to temporarily maintain two release line versions in parallel.

I think the need can be summarised as a plugin maintainer wants/needs to adopt semver (or similar versioning scheme) for their plugin alongside some level of backward support.

The workflow suggested above with short-term branches and a manually triggered GitHub action could handle this without causing too much noise/disruption to the repository or plugins that do not have this need.

[nickboldt]

From a real-life simple example we had recently:

• our plugin for RBAC support was updated to include changes in Backstage 1.27 (main branch)
• then a CVE was discovered which required applying a fix to that plugin and including it in our Backstage 1.26.5-based branch (1.2.2)
• however due to the changes in the 1.27 based version not being compatible with 1.26.5, we needed a way to patch the security issue in a version of the plugin that worked with 1.26.5 WHILE still developing new stuff based on 1.27.

The net result was we needed a BS 1.26-compliant RBAC plugin. This was achieved with two branches:

https://github.com/janus-idp/backstage-plugins/blob/1.2.x/plugins/rbac/package.json 1.20.11 https://github.com/janus-idp/backstage-plugins/blob/main/plugins/rbac/package.json 1.28.2

If we could ONLY move forward with a single main branch, this would have been impossible as the 1.21.z and newer versions in main included new features/changes that didn't work with 1.2 (BS 1.26.5).

TL;DR, our maintenance support requires fixing things in older releases as far back as 6mo ago, which can be achieved very simply with the ability to release community plugins from a temporary branch based on an older tag, as @BethGriggs proposes above.

We want to migrate our janus-idp/backstage-plugins content to the backstage community-plugins repo, but we need a way to support it once it's there with once-in-a-while manually-triggered maintenance releases from older y-streams of our plugins.

[Rugvip]

Yep, makes sense and we do need this for critical security fixes 👍

I'd push back on this though:

Other reasons may be tied to the service the plugin provides - a plugin that integrates with a service that has a new major versions with different APIs may prefer to temporarily maintain two release line versions in parallel.

It's likely always best to build support for those multiple versions into the plugin itself instead, that's gonna end up being simpler both for the maintainers and consumers of the plugin. I don't think we want to leave it to the one installing a plugin to have to select which version of the plugin to use, and on the maintainer side I don't think we want to have to manage applying changes multiple times.

[BethGriggs]

Sure, that could be our recommendation. Generally, in this project area, we’ve maintained that plugin owners have autonomy over their plugins and versioning decisions. In my view, if we can create a workflow that is generic enough to cover the main use cases for bug and security fixes, while also supporting plugin owners who want to manage in a more niche manner, I don't see any reason to dictate or pushback on how our community choose to version their plugins.

[nickboldt]

Another aspect of this is that it's not always possible to have a plugin support the latest runtime, due to incompatible dependency requirements. We have this problem already brewing with keycloak support as at SOME POINT we'll have a plugin that works with the latest Keycloak (rather than pinned to version 22) but we still have to do 6mo of bug/security maintenance even after we have a newer solution in main.

TL;DR, maintenance is required, and a simple workflow that allows this is necessary.

Would it be possible to grant some users access to manually publish plugins to the @backstage-community scope at npmjs.com? That's protected by 2FA and can be revoked at any time.

You can associate multiple people in a team, then associate a team with a subset of packages, which while it'll involve some adminstration, will provide the granular permission you want at the per-workspace (or per-team) level.

Would that be an acceptable approach to federating the permissions of workspace owners to control their own packages' maintenance lifecycles? If you need more info or screenshots on how this works, LMK and I can add.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[BethGriggs]

After a lot of discussion and iterations, we now have documented a process for doing interim releases in the Plugin Maintainer Guide using the recently added release_workspace_version.yml. The workflow ensures that interim releases go through the same PR approval and Version Packages steps with an appropriate audit trail.

I do now agree that we should expect the process only to be used to ship occasional interim releases rather than handling the parallel development of multiple release lines of the same plugin. If that level of complexity is necessary while the plugin is in active development then it's probably a sign that it would benefit from being developed in a dedicated repository.

I'll leave this open for now as a reminder that we need to document or automate the creation of the protected workspace branches with consistent rules/restrictions. As the number of plugins that have required this capability so far has been very small (~3 to my knowledge) it was agreed that initially this process could be ad-hoc/manual process.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[BethGriggs]

I'll leave this open for now as a reminder that we need to document or automate the creation of the protected workspace branches with consistent rules/restrictions. As the number of plugins that have required this capability so far has been very small (~3 to my knowledge) it was agreed that initially this process could be ad-hoc/manual process.

I've documented the branch settings in #3934. The need for this workflow has continued to be minimal, so I think we can then close this issue as we have an end-to-end process working and documented. We can consider adjustments and enhancements to the process should the need arise in future.