Changes & Experiments Strategy

[steveduffin]

There is a growing need for Toolkit to adapt to business changes, feedback and design patterns that are still being refined.

How do we accommodate styling that is at a less mature stage than other Toolkit UI but needs to live somewhere that's discoverable and shareable; a place that allows it live alongside Toolkit without creating the burden of frequent breaking changes for Toolkit's consumers who expect a stable code base?

This styling might include:

• New design patterns that we suspect are reusable across the estate but have yet to find their final form.
• Significant, breaking changes to existing components that need validation in a production environment.

By experimental we mean subject to frequent change; the goal being more freedom for faster, more frequent iteration to allow patterns to evolve. We would expect the same high level of code quality and testing and the standard peer review procedure to still apply.

Consumers should be given the option to opt in to these experiments and by doing so, can expect a certain level of volatility.

Once considered mature, these experimental styles should be submitted to toolkit as standard contributions.

Current leading Ideas:

1. Experimental branch / tag

Experiments could be PR'd into a dedicated branch or marked with a set tag which consumers then opt into with dependencies pointed at a git reference.

2. Naming convention

Effectively a flag that highlights certain styling should be consumed with caution e.g c-widget[alpha]. While this would help highlight the maturity of a given feature and allow features to live alongside the rest of toolkit, this manual approach could require a lot of maintenance.

3. Dedicated Package

We introduce a fourth package dedicated to experimental features e.g sky-toolkit-labs. It could contain a version of toolkit that houses the specific changes being worked on. In in simplest form, this could look like @import "~sky-toolkit-labs/components/new-widget"; Everytime new-widgetchanges, we would bump the version of toolkit but we don't use semvar; all changes to experiments are treated as minor bumps, even if breaking. That way we can still track changes to specific versions. When new-widget is mature, it's moved from the lab to toolkit-core|ui and consumers only need to update the import by substituting labs for the relevant path (assuming their app is up to date with the latest incarnation of the experiment).

[MrDinsdale]

Personally I'm not sure how well an experimental branch approach would work with platforms featuring multiple apps, where some might require separate experimental features.

I'm not opposed to a naming convention to separate unstable or experimental components, my concern however is that it clouds toolkit as the single source of truth perhaps? The maintenance could be a significant pain point for users.

I think a dedicated package (or even repo) gives these components sufficient isolation from our brand compliant ui while allowing safe iteration and development without compromising core and ui. I think it makes the most sense to start testing this approach first as it also has a minimal impact should we decide it isn't working out.

[steveduffin]

Toolkit Labs

Proposal

Experiments can be added into one of two protected "lab" branches

• Alpha: Contains breaking changes
• Beta: Non breaking changes

Breaking Change Workflow:

• Feature branch tkt-A is created
• tkt-A is identified as a breaking change by community or by author(*)
• A PR is raised to merge tkt-A to alpha
• PR is approved and merged following (adapted**) PR rules Design Contributions guide
• tkt-A is not deleted after merge.
• A maintainer prepares an alpha release
• alpha released with alpha-x suffix and tag e.g if toolkit master was at 3.20.0 it would become3.20.0-alpha-0:

  $ lerna publish --npm-tag=alpha
  $ > prerelease
  $ alpha
  

• Consumer runs npm install sky-toolkit@alpha to bump dependancy to latest alpha release e.g 3.20.0-alpha-0
• A further change is required for a feature
• Changes are made in tkt-A
• A new PR is made to merge tkt-A to alpha
• Steps are repeated as above
• Once tkt-A is considered ready for release, a PR is raised to merge tkt-A into develop
• Standard PR process is followed for a release. It should be mentioned on the PR that the change has gone through labs to allow for a more seamless integration.
• As part of the master release process, alpha branch should be reset so it has parity with master

None-breaking change

• Feature branch tkt-B is created
• tkt-B is identified as a none breaking change but still experimental(*)
• Same steps as breaking change, only using beta as branch, release id and tag

Notes

• Keeping labs up to date with develop could be a pain point. Is this mitigated by labs being effectively reset when toolkit master is released?
• Feature branches need to remain after merging to labs in case there are further updates required and labs is reset. They can be deleted once they get released into master.
• (*) We need clear docs on this step: what defines an experiment, and if it's to go in alpha or beta?
• (**) Contribution guidelines need to be updated to reflect requirements around the use of labs.
• If a lab branch has had other breaking changes merged in in-between releases, consumers will need to update their consuming app as it is vulnerable to other breaks. This has to be part of the contract with using labs. Once a change is mature, the consumer then has motivation to move feature out of labs into master to gain stability.
• How do we document / track what features are in labs at a given time? Do we update the relevant readme as part of the feature? Is the commit history enough?
• The standard flow of features to develop should remain. The use of labs is for anything that needs to be iterated upon after reaching live.

Useful reading:

http://carrot.is/coding/npm_prerelease https://semver.org/#spec-item-9 https://medium.com/@mbostock/prereleases-and-npm-e778fc5e2420

[lukewhitehouse]

Hey @steveduffin great work on getting this proposal together. I've been off so haven't had chance to look at this until now but I've got a couple of questions:

Alpha/beta branches:

When an alpha/beta branch is created (tkt-A or tkt-B respectively), would everyone use that branch for changes into that prerelease? i.e. hypothetically we're on version 4.2 of toolkit right now, let's say Bob creates a breaking change, so we create a tkt-A branch and a subsequent 5.0.0-alpha-0 branch. If Fred then comes along with his own breaking change, I understand that the alpha tag would now be 5.0.0-alpha-1, but what happens with the branch that it was created in. Does Fred also use Bob's branch, or does he create tkt-A-1 or something? And what would that branch be based off, master or tkt-A?

Tags and workflow

If a lab branch has had other breaking changes merged in in-between releases, consumers will need to update their consuming app as it is vulnerable to other breaks. This has to be part of the contract with using labs. Once a change is mature, the consumer then has motivation to move feature out of labs into master to gain stability.

To combat this, I would think we can get into a workflow where if someone is opting into an alpha branch that they then assume that any updates to alpha tags (i.e. 5.0.0-alpha-0 to 5.0.0-alpha-1) is a breaking change and to SPIKE the merge as such, if they need it. Baring in mind there may be points where they want changes from master so they may need to update to the latest alpha tag regardless or what other changes are being made.