Yarn PnP failures related to peer dependencies around client-sts

[ankon]

Checkboxes for prior research

• [X] I've gone through Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow.
• [X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

We've been using Yarn's PnP linking to speed up builds and generally try to enforce strict dependencies.

Recently we noticed that builds using esbuild failed when not marking the @aws-sdk packages as external:

$ yarn
➤ YN0000: ┌ Resolution step
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.590.0 doesn't provide @aws-sdk/client-sts (p20ccd), requested by @aws-sdk/credential-provider-web-identity
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.590.0 doesn't provide @aws-sdk/client-sts (p5d9f9), requested by @aws-sdk/credential-provider-ini
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @aws-sdk/client-sts (p0057b), requested by @aws-sdk/credential-provider-web-identity
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @aws-sdk/client-sts (pe8c2b), requested by @aws-sdk/credential-provider-ini
➤ YN0002: │ @aws-sdk/credential-provider-sso@npm:3.590.0 doesn't provide @aws-sdk/client-sso-oidc (p6168f), requested by @aws-sdk/token-providers
➤ YN0002: │ @aws-sdk/credential-provider-sso@npm:3.609.0 doesn't provide @aws-sdk/client-sso-oidc (p7c0eb), requested by @aws-sdk/token-providers
➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0000: └ Completed
➤ YN0000: Done with warnings in 0s 550ms

Building with AWS SDK v3 as external:

$ esbuild index.js --target=es2020 --bundle --platform=node --format=cjs --minify --outfile=build/index.js --external:@aws-sdk/*

  build/index.js  246.6kb

⚡ Done in 55ms

Building with AWS SDK v3 bundled into the result:

$ esbuild index.js --target=es2020 --bundle --platform=node --format=cjs --minify --outfile=build/index.js
✘ [ERROR] Could not resolve "@aws-sdk/client-sts"

    ../../.yarn/__virtual__/@aws-sdk-credential-provider-web-identity-virtual-ff1c7036c6/0/cache/@aws-sdk-credential-provider-web-identity-npm-3.609.0-a2a0bf35c3-7a95a6c479.zip/node_modules/@aws-sdk/credential-provider-web-identity/dist-cjs/fromWebToken.js:32:113:
      32 │         const { getDefaultRoleAssumerWithWebIdentity } = await Promise.resolve().then(() => __importStar(require("@aws-sdk/client-sts")));
         ╵                                                                                                                  ~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on "@aws-sdk/client-sts",
  but the package "@aws-sdk/client-sts" has not been installed:

    ../../.pnp.cjs:1998:36:
      1998 │             ["@aws-sdk/client-sts", null],\
           ╵                                     ~~~~

  You can mark the path "@aws-sdk/client-sts" as external to exclude it from the bundle, which will
  remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

✘ [ERROR] Could not resolve "@aws-sdk/client-sts"

    ../../.yarn/__virtual__/@aws-sdk-credential-provider-ini-virtual-3b149592d0/0/cache/@aws-sdk-credential-provider-ini-npm-3.609.0-ce49945e2f-c8a2e83ff8.zip/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/index.js:104:89:
      104 │     const { getDefaultRoleAssumer } = await Promise.resolve().then(() => __toESM(require("@aws-sdk/client-sts")));
          ╵                                                                                          ~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on "@aws-sdk/client-sts",
  but the package "@aws-sdk/client-sts" has not been installed:

    ../../.pnp.cjs:1684:36:
      1684 │             ["@aws-sdk/client-sts", null],\
           ╵                                     ~~~~

  You can mark the path "@aws-sdk/client-sts" as external to exclude it from the bundle, which will
  remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

✘ [ERROR] Could not resolve "@aws-sdk/client-sso-oidc"

    ../../.yarn/__virtual__/@aws-sdk-token-providers-virtual-0203582f7b/0/cache/@aws-sdk-token-providers-npm-3.609.0-044ab44290-e9cff1b949.zip/node_modules/@aws-sdk/token-providers/dist-cjs/index.js:51:79:
      51 │   const { SSOOIDCClient } = await Promise.resolve().then(() => __toESM(require("@aws-sdk/client-sso-oidc")));
         ╵                                                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on
  "@aws-sdk/client-sso-oidc", but the package "@aws-sdk/client-sso-oidc" has not been installed:

    ../../.pnp.cjs:2986:41:
      2986 │             ["@aws-sdk/client-sso-oidc", null],\
           ╵                                          ~~~~

  You can mark the path "@aws-sdk/client-sso-oidc" as external to exclude it from the bundle, which
  will remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

3 errors

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.11.1

Reproduction Steps

https://github.com/ankon/20240705-aws-sdk-v3-peer-deps:

Run yarn to reproduce peer dependency warnings; then yarn build to build using bundling or yarn build-external to build with the AWS SDK marked as external.

Observed Behavior

See description: peer-dependency warnings, build failures.

Expected Behavior

• yarn install properly installs without warnings.
• Builds work in both --external:@aws-sdk/* and bundled cases.

Possible Solution

No response

Additional Information/Context

This seems to come up a lot of times, our quick search found:

• https://github.com/aws/aws-sdk-js-v3/commit/5ce5600249cc163188e6fa8facf8ec7b8f435793#diff-f8a4e849dfd115c55571be87d8df85a22a0579ecf8bbef53ddccc972eb26bec1
• https://github.com/aws/aws-sdk-js-v3/issues/5518
• https://github.com/aws/aws-sdk-js-v3/issues/5818
• https://github.com/aws/aws-sdk-js-v3/commit/5ce5600249cc163188e6fa8facf8ec7b8f435793

[richa-misra]

Having the same issue with latest version aws-sdk version 3.624.0. Need to upgrade to use latest as it has change for fast-xml-parser but then fails on peer dependencies. Any options please?

[wojtekmaj]

If anything, you could use yarnrc.yml and packageExtensions to patch up incorrectly defined dependencies.

[wojtekmaj]

Here's are my error messages, a bit more detailed:

Package @aws-sdk/credential-provider-node@npm:3.622.0 is requested to provide @aws-sdk/client-sts by its descendants

@aws-sdk/credential-provider-node@npm:3.622.0
├─ @aws-sdk/credential-provider-ini@npm:3.622.0 [a1277] (via ^3.622.0)
│  └─ @aws-sdk/credential-provider-web-identity@npm:3.621.0 [a1277] (via ^3.621.0)
└─ @aws-sdk/credential-provider-web-identity@npm:3.621.0 [a1277] (via ^3.621.0)

✘ Package @aws-sdk/credential-provider-node@npm:3.622.0 does not provide @aws-sdk/client-sts.

Package @aws-sdk/credential-provider-sso@npm:3.622.0 is requested to provide @aws-sdk/client-sso-oidc by its descendants

@aws-sdk/credential-provider-sso@npm:3.622.0
└─ @aws-sdk/token-providers@npm:3.614.0 [aa9ff] (via ^3.614.0)

✘ Package @aws-sdk/credential-provider-sso@npm:3.622.0 does not provide @aws-sdk/client-sso-oidc.

[Luke-lsm]

As @wojtekmaj suggested, adding the dependencies in .yarnrc.yml will solve the issue temporarily while we wait for a fix, you can keep working through the missing dependency errors and adding them. For example my packageExtension is as follows:

nodeLinker: pnp

yarnPath: .yarn/releases/yarn-4.4.1.cjs

packageExtensions:
  "@aws-sdk/credential-provider-node@>=3.609.0":
    dependencies:
      "@aws-sdk/client-sts": "*"
  "@aws-sdk/credential-provider-sso@>=3.609.0":
    dependencies:
      "@aws-sdk/client-sso-oidc": "*"

[jacob-alford]

@Luke-lsm thank you!

[aqeelat]

@Luke-lsm thank you We ended up using:

packageExtensions:
  "@aws-sdk/credential-provider-node@>=3.609.0":
    dependencies:
      "@aws-sdk/client-sts": "*"
  "@aws-sdk/credential-provider-sso@>=3.609.0":
    dependencies:
      "@aws-sdk/client-sso-oidc": "*"
      "@aws-sdk/client-sts": "*"

[aqeelat]

Btw, the issue not limited to pnp. We're using node_modules and still have the issue.

[jacob-alford]

Yarn is just too darn reliable

[kuhe]

This should be fixed as of https://github.com/aws/aws-sdk-js-v3/releases/tag/v3.735.0.

yarn: 4.6.0

with @aws-sdk/[email protected]:

yarn explain peer-requirements
p51e63 → ✓ @aws-sdk/client-ssm@npm:3.609.0 provides @aws-sdk/client-sts@npm:3.609.0 to @aws-sdk/client-sso-oidc@npm:3.609.0 [724a8]
p70f73 → ✓ @aws-sdk/client-ssm@npm:3.609.0 doesn't provide @types/aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
p5594f → ✓ @aws-sdk/client-ssm@npm:3.609.0 doesn't provide @types/aws-sdk__client-sts to @aws-sdk/client-sso-oidc@npm:3.609.0 [724a8]
pae46c → ✓ @aws-sdk/client-ssm@npm:3.609.0 doesn't provide aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
pc4312 → ✓ @aws-sdk/client-sso-oidc@npm:3.609.0 [724a8] doesn't provide @types/aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
p72142 → ✓ @aws-sdk/client-sso-oidc@npm:3.609.0 [724a8] doesn't provide aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
pbfd02 → ✓ @aws-sdk/client-sso@npm:3.609.0 doesn't provide @types/aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
pe6bed → ✓ @aws-sdk/client-sso@npm:3.609.0 doesn't provide aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
p289f9 → ✓ @aws-sdk/client-sts@npm:3.609.0 provides @aws-sdk/client-sts@npm:3.609.0 to @aws-sdk/client-sso-oidc@npm:3.609.0 [724a8]
pfd009 → ✓ @aws-sdk/client-sts@npm:3.609.0 doesn't provide @types/aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
p6f0b0 → ✓ @aws-sdk/client-sts@npm:3.609.0 doesn't provide @types/aws-sdk__client-sts to @aws-sdk/client-sso-oidc@npm:3.609.0 [724a8]
p4d33d → ✓ @aws-sdk/client-sts@npm:3.609.0 doesn't provide aws-crt to @aws-sdk/util-user-agent-node@npm:3.609.0 [514b9]
pcafcc → ✘ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @aws-sdk/client-sts to @aws-sdk/credential-provider-ini@npm:3.609.0 [9a531] and 1 other dependency
p8cb99 → ✓ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @types/aws-sdk__client-sts to @aws-sdk/credential-provider-ini@npm:3.609.0 [9a531] and 1 other dependency
p2326d → ✘ @aws-sdk/credential-provider-sso@npm:3.609.0 doesn't provide @aws-sdk/client-sso-oidc to @aws-sdk/token-providers@npm:3.609.0 [b85bf]
pbc6ea → ✓ @aws-sdk/credential-provider-sso@npm:3.609.0 doesn't provide @types/aws-sdk__client-sso-oidc to @aws-sdk/token-providers@npm:3.609.0 [b85bf]

with @aws-sdk/[email protected]:

p2ee80 → ✓ @aws-sdk/client-ssm@npm:3.735.0 doesn't provide @types/aws-crt to @aws-sdk/util-user-agent-node@npm:3.734.0 [0d053]
p0828c → ✓ @aws-sdk/client-ssm@npm:3.735.0 doesn't provide aws-crt to @aws-sdk/util-user-agent-node@npm:3.734.0 [0d053]
p2a227 → ✓ @aws-sdk/client-sso@npm:3.734.0 doesn't provide @types/aws-crt to @aws-sdk/util-user-agent-node@npm:3.734.0 [0d053]
p090d8 → ✓ @aws-sdk/client-sso@npm:3.734.0 doesn't provide aws-crt to @aws-sdk/util-user-agent-node@npm:3.734.0 [0d053]
pbdb6a → ✓ @aws-sdk/nested-clients@npm:3.734.0 doesn't provide @types/aws-crt to @aws-sdk/util-user-agent-node@npm:3.734.0 [0d053]
paea75 → ✓ @aws-sdk/nested-clients@npm:3.734.0 doesn't provide aws-crt to @aws-sdk/util-user-agent-node@npm:3.734.0 [0d053]

[wojtekmaj]

I'm very happy to confirm. peerDependency warnings are gone after I ran yarn up "@aws-sdk/*" :)

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.