Dependency conflict after releasing a breaking change.

[sobolk]

From https://github.com/aws-amplify/amplify-js-samples-staging/pull/765#discussion_r1436674841 .

It appears that releasing a breaking change to @aws-amplify/plugin-types does not propagate correctly to @aws-amplify/backend-cli that also depends on this transitively.

Note that breaking change in @aws-amplify/plugin-types was released with proper version increment, i.e. minor version bump in 0.x versioning mode, (see https://dev.to/malykhinvi/semver-0-x-x-46e7 for reference).

Log from failure from an app before PR above was merged. The app didn't use package lock file

/amplify/backend.ts:6:3
Type error: Type 'ConstructFactory<AmplifyAuth>' is not assignable to type 'ConstructFactory<Construct>'.
  Types of property 'getInstance' are incompatible.
    Type '(props: ConstructFactoryGetInstanceProps) => AmplifyAuth' is not assignable to type '(props: ConstructFactoryGetInstanceProps) => Construct'.
      Types of parameters 'props' and 'props' are incompatible.
        Type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/backend/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps' is not assignable to type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps'.
          The types returned by 'constructContainer.getConstructFactory(...)' are incompatible between these types.
            Type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/backend/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory<T> | undefined' is not assignable to type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory<T> | undefined'.
              Type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/backend/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory<T>' is not assignable to type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory<T>'.
                Types of property 'getInstance' are incompatible.
                  Type '(props: import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/backend/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps) => T' is not assignable to type '(props: import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps) => T'.
                    Types of parameters 'props' and 'props' are incompatible.
                      Type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps' is not assignable to type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/backend/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps'.
                        The types of 'constructContainer.registerConstructFactory' are incompatible between these types.
                          Type '(token: string, provider: import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory) => void' is not assignable to type '(token: string, provider: import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/backend/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory) => void'.
                            Types of parameters 'provider' and 'provider' are incompatible.
                              Type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/backend/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory' is not assignable to type 'import("/Users/zheallan/workspace/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactory'.
                                The types returned by 'getInstance(...)' are incompatible between these types.
                                  Type 'unknown' is not assignable to type 'ResourceProvider'.

  4 |
  5 | defineBackend({
> 6 |   auth,
    |   ^
  7 |   data,
  8 | });
  9 |
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Reproduction steps

1. Checkout this commit https://github.com/aws-amplify/amplify-js-samples-staging/commit/9662f904e55b5cfd8b52ee4d22ddcd98797d794f . (the commit after it fixes the problem).
2. cd samples/next/api/data-client-gen2
3. npm install
4. npx sandbox

Result:

bcd0740fa72c:data-client-gen2 sobkamil$ npm install
npm WARN deprecated @babel/[email protected]: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-class-properties instead.
npm WARN deprecated @babel/[email protected]: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-object-rest-spread instead.
npm WARN deprecated [email protected]: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.

added 1067 packages, and audited 1182 packages in 31s

178 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
bcd0740fa72c:data-client-gen2 sobkamil$ npx amplify sandbox
[Sandbox] Initializing...
[Sandbox] Executing command `deploy`
amplify/backend.ts(6,3): error TS2322: Type 'ConstructFactory<AmplifyAuth>' is not assignable to type 'ConstructFactory<Construct>'.
  Types of property 'getInstance' are incompatible.
    Type '(props: ConstructFactoryGetInstanceProps) => AmplifyAuth' is not assignable to type '(props: ConstructFactoryGetInstanceProps) => Construct'.
      Types of parameters 'props' and 'props' are incompatible.
        Type 'import("/Users/sobkamil/git/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps' is not assignable to type 'import("/Users/sobkamil/git/amplify-js-samples-staging/samples/next/api/data-client-gen2/node_modules/@aws-amplify/auth-construct-alpha/node_modules/@aws-amplify/plugin-types/lib/construct_factory").ConstructFactoryGetInstanceProps'.
          The types of 'constructContainer.getOrCompute' are incompatible between these types.
            Type '(generator: ConstructContainerEntryGenerator) => Construct' is not assignable to type '(generator: ConstructContainerEntryGenerator<object>) => ResourceProvider'.
              Types of parameters 'generator' and 'generator' are incompatible.
                Type 'ConstructContainerEntryGenerator<object>' is not assignable to type 'ConstructContainerEntryGenerator'.
                  The types returned by 'generateContainerEntry(...)' are incompatible between these types.
                    Property 'node' is missing in type 'ResourceProvider<object>' but required in type 'Construct'.
TypeScript validation check failed, check your backend definition
Caused By:
[Sandbox] Watching for file changes...
^C[Sandbox] Shutting down
? Would you like to delete all the resources in your sandbox environment (This cannot be undone)? y
[Sandbox] Deleting all the resources in the sandbox environment...
[Sandbox] Executing command `destroy`
amplify-dataclientgen2-sobkamil-sandbox-846a36cb14: destroying... [1/1]

 ✅  amplify-dataclientgen2-sobkamil-sandbox-846a36cb14: destroyed

[Sandbox] Finished deleting.

[sobolk]

After taking another look at the release content https://github.com/aws-amplify/amplify-backend/pull/791 it seems that we released some packages with patch and some with minor version increment. Therefore, for example the ^0.x.y (special versioning) pulled automatically new @aws-amplify/auth-construct-alpha which had only patch increment, however it pulled new minor version (0.x breaking change) of @aws-amplify/plugin-types and led to inconsistent dependency graph.

Similar scenario may happen when we GA and switch to ^x.y.z versioning. Therefore it seems that releasing a breaking change of certain packages like @aws-amplify/plugin-types should bubble up though deps graph and force more aggressive version increment (appropriate for breaking change) of all packages that depend on such package.

We should:

1. Identify which "core" packages should propagate major version upgrade across our packages
2. Implement enforcement (PR check?)

[sobolk]

Note:

When we graduate from 0.x versions this issue might be somewhat mitigated by the fact that if we rolled major version of plugin-types and mix of minor and patch increments elsewhere then all of them would be automatically absorbed in customer's app and have new major version of plugin-types consistently pulled into dependency graph.

This should reduce potential impact to cases when:

1. Customer is pinning version of one of components.
2. Customer is explicitly depending on plugin-types .

Therefore, we should consider holding off with this to see what's actual impact is and how strict should we be with versioning of this kind of changes.