yarn 3 ignores npmAuthTokens in ~/.yarnrc.yml: "Invalid authentication (as an anonymous user)" when using private repo, despite `yarn npm login`

[gustafc]

TL;DR: yarn 3.2.0 seems to ignore auth tokens in $HOME/.yarnrc.yml, since the token generated by yarn npm login has to be manually added to $PROJECT/.yarnrc.yml to authenticate when doing yarn install. There's a reproduction script at the end!

I'm trying to upgrade a project from yarn 1.x to modern yarn using the migration checklist, and I'm running into trouble at the step where I run yarn install:

1. Run npm install -g yarn to update the global yarn version to latest v1
2. Go into your project directory
3. Run yarn set version berry to enable v2 (cf Install for more details)
4. If you used .npmrc or .yarnrc, you'll need to turn them into the new format (see also 1, 2)
5. Add nodeLinker: node-modules in your .yarnrc.yml file
6. Commit the changes so far (yarn-X.Y.Z.js, .yarnrc.yml, ...)
7. Run yarn install to migrate the lockfile

We install all modules from a private Nexus repository, which houses our private packages while also acting as a proxy to the public NPM repo. This Nexus repo requires authentication at all times, which is where the troubles begin.

This is the project's .yarnrc.yml:

yarnPath: .yarn/releases/yarn-3.2.0.cjs
nodeLinker: node-modules
npmRegistryServer: "https://nexus.example.com/repository/npm-all"
npmAlwaysAuth: true
npmRegistries:
  "https://nexus.example.com/repository/npm-all":
    npmAlwaysAuth: true

If I try to install without logging in, it predictably fails since I haven't logged in to the Nexus server:

$ yarn install
➤ YN0000: ┌ Resolution step
➤ YN0041: │ @testing-library/jest-dom@npm:5.16.4: Invalid authentication (as an anonymous user)
➤ YN0000: └ Completed in 0s 513ms
➤ YN0000: Failed with errors in 0s 521ms

So I login:

$ yarn npm login
➤ YN0000: Logging in to https://nexus.example.com/repository/npm-all

✔ Username: · my-name
✔ Password: · ****************

➤ YN0000: Successfully logged in
➤ YN0000: Done in 17s 175ms

Seems to have gone well, and it has generated a $HOME/.yarnrc.yml:

npmRegistries:
  "https://nexus.example.com/repository/npm-all":
    npmAuthToken: NpmToken.00000000-1111-2222-3333-444444444444

But yarn install still fails, and I actually don't seem to be logged in:

$ yarn install
➤ YN0000: ┌ Resolution step
➤ YN0041: │ @testing-library/jest-dom@npm:5.16.4: Invalid authentication (as an anonymous user)
➤ YN0000: └ Completed in 0s 523ms
➤ YN0000: Failed with errors in 0s 532ms

$ yarn npm whoami
➤ YN0033: No authentication configured for request
➤ YN0000: Failed with errors in 0s 4ms

The only workaround I've found is copying the npmAuthToken from $HOME/.yarnrc.yml to $PROJECT/.yarnrc.yml:

$ cat $PROJECT/.yarnrc.yml
yarnPath: .yarn/releases/yarn-3.2.0.cjs
nodeLinker: node-modules
npmRegistryServer: "https://nexus.example.com/repository/npm-all"
npmAlwaysAuth: true
npmRegistries:
  "https://nexus.example.com/repository/npm-all":
    npmAuthToken: NpmToken.00000000-1111-2222-3333-444444444444
    npmAlwaysAuth: true

$ yarn npm whoami
➤ YN0000: my-name
➤ YN0000: Done in 0s 74ms

This is obviously not something I want to do. How can I tell yarn to use the auth token that yarn npm login creates?

Versions used:

$ yarn -v
3.2.0
$ node -v
v14.16.0

EDIT: So I managed to set up a script that reproduces this issue. It starts a new Nexus server through docker, sets up yarn project, logs in, and tries to add a package.

#!/bin/bash
set -exuo pipefail

UNIQUE_STRING=$(date '+%Y%m%d-%H%M%S')
PROJECT=/tmp/yarn3-auth-repro-$UNIQUE_STRING
# Step 1: Create a new project in /tmp
mkdir -p $PROJECT
cd $_
yarn init -2
echo '
unsafeHttpWhitelist:
  - localhost
npmRegistryServer: "http://localhost:8081/repository/npm-all"
npmAlwaysAuth: true
' >>.yarnrc.yml

# Step 2: Set up Nexus
NEXUS_CONTAINER=nexus-repro-container-$UNIQUE_STRING

docker run --rm --detach -p 8081:8081 --name $NEXUS_CONTAINER sonatype/nexus3@sha256:66fe12b1eb3e97bae72eb3c2c4e436499d41ff144cdfd1dcd0718df738304732
function cleanup {
  read -p "Stop Nexus docker container $NEXUS_CONTAINER? (y/n) " STOP_CONTAINER
  if [[ $STOP_CONTAINER == y* ]]; then
    docker stop $NEXUS_CONTAINER
  else
    echo "You can play around in the project by going to the following directory:"
    echo "  cd $PROJECT"
    echo "To stop the Nexus container, run:"
    echo "  docker stop $NEXUS_CONTAINER"
  fi
}
trap cleanup EXIT

while ! docker logs $NEXUS_CONTAINER | grep 'Started Sonatype Nexus OSS' ; do echo "Waiting for $NEXUS_CONTAINER to start..."; sleep 2; done
printf '\a'
NEXUS_USER=admin
NEXUS_PASS="$(docker exec $NEXUS_CONTAINER cat /nexus-data/admin.password)"
# # Change admin password
# curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/rest/internal/ui/onboarding/change-admin-password' -X PUT --data-raw 'admin123'
# sleep 1
# NEXUS_PASS=admin123
# Disallow anonymous access
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_AnonymousSettings","method":"update","data":[{"enabled":false,"userId":"anonymous","realmName":"NexusAuthorizingRealm"}],"type":"rpc","tid":14}'

# Create npm repo proxy
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_Repository","method":"create","data":[{"attributes":{"npm":{"removeNonCataloged":false,"removeQuarantinedVersions":false},"proxy":{"remoteUrl":"https://registry.npmjs.org","contentMaxAge":1440,"metadataMaxAge":1440},"httpclient":{"blocked":false,"autoBlock":true,"connection":{"useTrustStore":false}},"storage":{"blobStoreName":"default","strictContentTypeValidation":true},"negativeCache":{"enabled":true,"timeToLive":1440},"cleanup":{"policyName":[]}},"name":"npm-proxy","format":"","type":"","url":"","online":true,"routingRuleId":"","authEnabled":false,"httpRequestSettings":false,"recipe":"npm-proxy"}],"type":"rpc","tid":31}'

# Create npm repo group (which is my real setup)
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_Repository","method":"create","data":[{"attributes":{"storage":{"blobStoreName":"default","strictContentTypeValidation":true},"group":{"memberNames":["npm-proxy"]}},"name":"npm-all","format":"","type":"","url":"","online":true,"recipe":"npm-group"}],"type":"rpc","tid":44}'

# Make NPM login work <https://issues.sonatype.org/browse/NEXUS-20170>
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_RealmSettings","method":"update","data":[{"realms":["NexusAuthenticatingRealm","NexusAuthorizingRealm","NpmToken"]}],"type":"rpc","tid":43}'


# Step 3: Login
(echo "$NEXUS_USER"; sleep 2; echo "$NEXUS_PASS") | yarn npm login
yarn npm whoami

# Step 4: Add any random package to make sure auth works (this will fail)
yarn add mkdirp@latest

If you run this script and don't shut down Nexus at the end (you will be asked if you want to), you can cd to the directory the script created, and manually add the token from $HOME/.yarnrc.yml to $PROJECT/.yarnrc.yml, like so:

unsafeHttpWhitelist:
  - localhost
npmRegistryServer: "http://localhost:8081/repository/npm-all"
npmAlwaysAuth: true
npmRegistries:
  "http://localhost:8081/repository/npm-all":
    npmAlwaysAuth: true
    npmAuthToken: NpmToken.xxxxxxxxxxxxx

[gustafc]

Given the lack of reactions to this issue, I've added a Bash script to the original post which reproduces the error, using a local Nexus running in Docker. Enjoy!

[HeatherFlux]

I'm feeling your pain on this one as well. It seems to lose its auth credentials when dealing with non root packages in my monorepo.

[rosskevin]

I am transitioning our projects to v3, and I have found the failure in authentication to the github package repo.

nodeLinker: pnp

npmScopes:
  alienfast:
    npmAlwaysAuth: true
    npmRegistryServer: "https://npm.pkg.github.com"

yarnPath: .yarn/releases/yarn-3.2.0.cjs

~/p/hello-world-yarn ❯❯❯ yarn npm publish
➤ YN0000: .editorconfig
➤ YN0000: Readme.md
➤ YN0000: bin.js
➤ YN0000: example.gif
➤ YN0000: lib/index.js
➤ YN0000: package.json
➤ YN0000: Package archive published
➤ YN0000: Done in 2s 773ms
~/p/hello-world-yarn ❯❯❯ npm whoami
rosskevin
~/p/hello-world-yarn ❯❯❯ yarn npm whoami
➤ YN0033: No authentication configured for request
➤ YN0000: Failed with errors in 0s 4ms

As you can see, I can publish a private package, but yarn npm whoami reports no auth whereas npm whoami reports my rosskevin user.

I investigated this path because previous packages published on https://npm.pkg.github.com/ are reporting unavailable with misleading messages such as:

~/p/hello-world-yarn ❯❯❯ yarn add @alienfast/eslint-config
➤ YN0000: ┌ Resolution step
➤ YN0033: │ eslint-config-prettier@npm:^8.5.0: No authentication configured for request
➤ YN0000: └ Completed in 3s 875ms
➤ YN0000: Failed with errors in 3s 880ms

whereas npm works fine:

~/p/hello-world-yarn ❯❯❯ npm add @alienfast/eslint-config                                                                                                                                                            ✘ 1 

added 195 packages, and audited 196 packages in 13s

64 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

TL;DR yarn 3 authentication is not working with older github artifacts, and yarn npm whoami is misreporting No authentication configured for request even though yarn npm publish successfully publishes to the github repository!

~/p/hello-world-yarn ❯❯❯ yarn npm whoami --scope alienfast                                                                                                                                                           ✘ 1 
➤ YN0000: rosskevin
➤ YN0000: Done in 0s 348ms

(as an aside, I tried publishing a current @alienfast/eslint-config with yarn npm publish which is also successful but no yarn command can pull from that repo, even though npm can).

SO ... WHY can I publish but not install from the same config?

[rosskevin]

Bizarrely, yarn config shows opposite of what I have scoped:

➤ YN0000: npmRegistries                 Map(1) { 'https://npm.pkg.github.com' => Map(3) { 'npmAlwaysAuth' => false, 'npmAuthIdent' => null, 'npmAuthToken' => '********' } }
➤ YN0000: npmRegistryServer             'https://registry.yarnpkg.com'
➤ YN0000: npmScopes                     Map(1) { 'alienfast' => Map(6) { 'npmAlwaysAuth' => true, 'npmAuthIdent' => null, 'npmAuthToken' => null, 'npmAuditRegistry' => null, 'npmPublishRegistry' => null, 'npmRegistryServer' => 'https://npm.pkg.github.com' } }
➤

npmAlwaysAuth is clearly set to true under the alienfast scope IN the project's yarnrc.yml.

[rosskevin]

Ok, so this seems to look like something attempted to be solved by the deep merge PR from #2106 which was rejected due to major breaking changes?

This is a bug, confirmed. The only way I can (so far) get yarn config to appear properly and yarn add to work properly is to include my token in the project's yarnrc.yml (huge security hole) and specify my auth with BOTH the npmRegistries key AND npmScopes.

e.g.

nodeLinker: pnp

npmRegistries:
  "https://npm.pkg.github.com":
    npmAuthToken: xxxxx
    npmAlwaysAuth: true

npmScopes:
  alienfast:
    npmAlwaysAuth: true
    npmRegistryServer: "https://npm.pkg.github.com"

yarnPath: .yarn/releases/yarn-3.2.0.cjs

^^^ this works as a project yarnrc.yml but is unusable from a security perspective.

[gustafc]

For the record, I've been trying to get this working in Sherlock, but can't really figure out how to pass username/password to yarn npm login. The shell trick (echo "$NEXUS_USER"; sleep 2; echo "$NEXUS_PASS") | yarn npm login which I use above doesn't seem to work when running in Sherlock.

[alamothe]

I found a solution in another issue. It is required to specify the token inside npmRegistries field. So your ~/.yarnrc.yml should look like:

npmRegistries:
  "https://my-registry-url":
    npmAuthToken: ...

npmScopes:
  myscope:
    npmAuthToken: ...

Not sure if both are required, but the first one certainly is.

[gustafc]

@alamothe this issue is called "yarn 3 ignores npmAuthTokens in ~/.yarnrc.yml" :) Running yarn npm login adds the token to ~/.yarnrc.yml, so that's not a problem; the problem is that yarn doesn't use the token from ~/.yarnrc.yml´ when I run yarn install`.

Don't know if adding npmScopeschanges anything, but we don't use scopes so that's an academic question. We use a Nexus server which both acts as proxy for the public NPM repo and serves our own private packages, no scopes involved, auth always required.

[alamothe]

@gustafc Interesting. It did ignore my token in ~/.yarnrc.yml if I only used npmScopes, but once I added npmRegistries too, it started to use it.

I definitely agree that there is a bug, I just wanted to help with a potential workaround until it is solved.

[arcanis]

It doesn't ignore the token, it just doesn't support configuration merging. In other words, the npmRegistries you declare in your project yarnrc overrides the one in the global one (same for npmScopes).

That's something we'll fix in the next major, since it's a breaking change, but it's quite tricky to implement in a satisfying way, so it hasn't been started yet.

[gustafc]

@arcanis I see. I eagerly await 4.0 then!

[alamothe]

Wow, so the reason it works for people who use scopes is that it can take npmScopes from project .yarnrc.yml and npmRegistries from home dir .yarnrc.yml. This way it is possible to separate scopes/registries configuration from tokens.

I don't see a workaround for people who don't use scopes.

@arcanis IMHO the only practical reason for separating the config is auth tokens, so perhaps that could simplify the solution i.e. it doesn't really need to be general.

[vocko]

It doesn't ignore the token, it just doesn't support configuration merging. In other words, the npmRegistries you declare in your project yarnrc overrides the one in the global one (same for npmScopes).

That's something we'll fix in the next major, since it's a breaking change, but it's quite tricky to implement in a satisfying way, so it hasn't been started yet.

@arcanis Is there an issue for the configuration merging that can be tracked? Or should I create one? It is quite a roadblock for us and we are very keen to get this feature in.

[mikelpr]

it doesn't work for me with scoped packages on yarn 3.2.3 @__@

having a .yarnrc.yml like this:

yarnPath: .yarn/releases/yarn-3.2.3.cjs
nodeLinker: node-modules
npmRegistries:
  "https://npm.pkg.github.com":
    npmAlwaysAuth: true
npmScopes:
  pkgscope:
    npmAlwaysAuth: true
    npmRegistryServer: "https://npm.pkg.github.com"

I still get ➤ YN0041: │ @pkgscope/package-name@npm:^0.1.0: Invalid authentication (as an anonymous user) on yarn add

[tim-rellify]

@arcanis Can we expect this change for v3 or will this just be included in v4? Because it isn't working in 3.3.1 for scoped packages.

project's .yarnrc.yml:

npmScopes:
  myscope:
    npmRegistryServer: "https://npm.pkg.github.com"
    npmAlwaysAuth: true

Login via yarn npm login -s myscope results in ~/.yarnrc.yml:

nodeLinker: node-modules

npmScopes:
  myscope:
    npmAuthToken: my-secret-auth-token

But installing a package from "myscope" in the exampke above results in "YN0041: Invalid authentication (as an anonymous user)". As a workaround, it is working with the following ~/.yarnrc.yml (no changes to the project's .yarnrc.yml:

nodeLinker: node-modules

npmRegistries:
  https://npm.pkg.github.com:
    npmAuthToken: my-secret-auth-token

But then I have to do manual work, because yarn npm login doesn't support logging in into a specific repository via parameter.

[arcanis]

It'll not be backported to 3.x considering it's a breaking change, and very difficult to cherry pick anyway. If you need it, please use the RCs.

[cowwoc]

Can someone please summarize what has to go in project/.yarnrc.yml and ~/.yarnrc.yml in version 3.x to publish without including the authentication tokens as part of the project?

I tried the following and it did not work:

project/.yarnrc.yml:

npmScopes:
  public:
    npmRegistryServer: "https://registry.npmjs.org"
    npmAlwaysAuth: true

~/.yarnrc.yml

npmRegistries:
  "https://registry.npmjs.org":
    npmAuthToken: xxx

I still get YN0033: No authentication configured for request when I run yarn npm publish --access=public

[cowwoc]

I figured it out. I needed to add npmPublishRegistry: "https://registry.npmjs.org" to ~/.yarnrc.yml .

[in-ch]

I had the same problem and solved it like this.

to ~/.yarnrc.yml

yarnPath: .yarn/releases/yarn-3.4.1.cjs
nodeLinker: node-modules
npmScopes:
  your-company:
    npmAlwaysAuth: true
    npmRegistryServer: "https://nexus.your-company.io/repository/your-npm/"
    npmAuthIdent: "username:password"

https://yarnpkg.com/configuration/yarnrc#npmAuthIdent

[mikelpr]

well this is problematic and commenters don't quite grok what's happening so hopefully this explanation clears it up:

if you have a npmScopes: your-company: block in your ~/.yarnrc.yml, any keys there will be completely ignored (not merged) if there's also a npmScopes: your-company: block on your project's .yarnrc.yml. Therefore, whatever you try to use inside the npmScopes: your-company: block in ~/.yarnrc.yml won't work at all. It will only work if there's no npmScopes: your-company: block in your project's .yarnrc.yml

yarn 4 fixes this no merging issue and will apply the project's .yarnrc.yml over ~/.yarnrc.yml but keeping the values of keys that are specified in ~/.yarnrc.yml but not in the project's .yarnrc.yml. so if you have a npmAlwaysAuth: true for the your-company scope on ~/.yarnrc.yml and not in the project's .yarnrc.yml where you have other keys for the your-company scope, on yarn 4 it will respect the npmAlwaysAuth from ~/.yarnrc.yml, while on yarn 3 it will be false as it is the default and it ignores the your-company scope block from your home's .yarnrc.yml because no merging.

hope this helps anyone and hope yarn 4 comes out soon officially.

[AdamGerthel]

For anyone using v3 still and looking for a solution, you can use env variables as a workaround, eg:

npmAuthToken: '${SOME_TOKEN:-}' // :- is added as a fallback to prevent a missing token from causing issues when the variable isn't necessary

and then use it:

SOME_TOKEN=<your token here> yarn npm publish

and of course, since it's an env variable, you can use other ways of loading it.

[asingh072318]

I am on yarn 3.5.1 have the local .yarnrc.yml defined like this:

`nodeLinker: node-modules

plugins:

• path: .yarn/plugins/plugin-dedupe-on-install.cjs spec: "https://raw.githubusercontent.com/ambar/yarn-plugin-dedupe-on-install/main/index.js"
• path: .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs spec: "@yarnpkg/plugin-workspace-tools"

npmRegistries: "REGISTRY URL": npmAuthToken: "TOKEN" npmAlwaysAuth: true

npmScopes: platform-ui: npmRegistryServer: REGISTRY URL npmAlwaysAuth: true npmAuthToken: 'TOKEN'

yarnPath: .yarn/releases/yarn-3.5.1.cjs`

And i am still getting this error:

`➤ YN0027: @platform-ui/design-system@unknown can't be resolved to a satisfying range ➤ YN0041: Invalid authentication (as an unknown user)

➤ Errors happened when preparing the environment required to run this command.`

[AdamGerthel]

@asingh072318 I had the same problem just now. Upgrading to Yarn v4 solved it.

[TheYorickable]

Ran into this issue while using a Sonatype Nexus Repository and the fix was using new token formatting.

npmAuthToken: https://nexus.intern.***.***.***/repository/npm-group/:authToken=XXXX-XXXXX-XXX

[email protected]

[gustafc]

To anyone running into this problem, the solution to the problem I had was:

1. Using yarn 4 (since yarn 4 tries to merge configs from ~/.yarnrc.yml and $PROJECT/.yarnrc.yml)
2. Logging in with yarn npm login --always-auth (since yarn 4 isn't that good at merging configs, this adds npmAlwaysAuth to the npmAuthToken["https://nexus.example.com/repository/npm-all"] entry in ~/.yarnrc.yml which would otherwise default to false and override the top-level npmAlwaysAuth in $PROJECT/.yarnrc.yml)

I'd still argue it's a bug that project-specific npmAlwaysAuth doesn't influence the behaviour of yarn 4, and thus 👉 #6097