berry
berry copied to clipboard
yarn 3 ignores npmAuthTokens in ~/.yarnrc.yml: "Invalid authentication (as an anonymous user)" when using private repo, despite `yarn npm login`
TL;DR: yarn 3.2.0 seems to ignore auth tokens in $HOME/.yarnrc.yml
, since the token generated by yarn npm login
has to be manually added to $PROJECT/.yarnrc.yml
to authenticate when doing yarn install
. There's a reproduction script at the end!
I'm trying to upgrade a project from yarn 1.x to modern yarn using the migration checklist, and I'm running into trouble at the step where I run yarn install
:
- Run npm install -g yarn to update the global yarn version to latest v1
- Go into your project directory
- Run yarn set version berry to enable v2 (cf Install for more details)
- If you used .npmrc or .yarnrc, you'll need to turn them into the new format (see also 1, 2)
- Add nodeLinker: node-modules in your .yarnrc.yml file
- Commit the changes so far (yarn-X.Y.Z.js, .yarnrc.yml, ...)
- Run yarn install to migrate the lockfile
We install all modules from a private Nexus repository, which houses our private packages while also acting as a proxy to the public NPM repo. This Nexus repo requires authentication at all times, which is where the troubles begin.
This is the project's .yarnrc.yml
:
yarnPath: .yarn/releases/yarn-3.2.0.cjs
nodeLinker: node-modules
npmRegistryServer: "https://nexus.example.com/repository/npm-all"
npmAlwaysAuth: true
npmRegistries:
"https://nexus.example.com/repository/npm-all":
npmAlwaysAuth: true
If I try to install without logging in, it predictably fails since I haven't logged in to the Nexus server:
$ yarn install
➤ YN0000: ┌ Resolution step
➤ YN0041: │ @testing-library/jest-dom@npm:5.16.4: Invalid authentication (as an anonymous user)
➤ YN0000: └ Completed in 0s 513ms
➤ YN0000: Failed with errors in 0s 521ms
So I login:
$ yarn npm login
➤ YN0000: Logging in to https://nexus.example.com/repository/npm-all
✔ Username: · my-name
✔ Password: · ****************
➤ YN0000: Successfully logged in
➤ YN0000: Done in 17s 175ms
Seems to have gone well, and it has generated a $HOME/.yarnrc.yml
:
npmRegistries:
"https://nexus.example.com/repository/npm-all":
npmAuthToken: NpmToken.00000000-1111-2222-3333-444444444444
But yarn install
still fails, and I actually don't seem to be logged in:
$ yarn install
➤ YN0000: ┌ Resolution step
➤ YN0041: │ @testing-library/jest-dom@npm:5.16.4: Invalid authentication (as an anonymous user)
➤ YN0000: └ Completed in 0s 523ms
➤ YN0000: Failed with errors in 0s 532ms
$ yarn npm whoami
➤ YN0033: No authentication configured for request
➤ YN0000: Failed with errors in 0s 4ms
The only workaround I've found is copying the npmAuthToken
from $HOME/.yarnrc.yml
to $PROJECT/.yarnrc.yml
:
$ cat $PROJECT/.yarnrc.yml
yarnPath: .yarn/releases/yarn-3.2.0.cjs
nodeLinker: node-modules
npmRegistryServer: "https://nexus.example.com/repository/npm-all"
npmAlwaysAuth: true
npmRegistries:
"https://nexus.example.com/repository/npm-all":
npmAuthToken: NpmToken.00000000-1111-2222-3333-444444444444
npmAlwaysAuth: true
$ yarn npm whoami
➤ YN0000: my-name
➤ YN0000: Done in 0s 74ms
This is obviously not something I want to do. How can I tell yarn to use the auth token that yarn npm login
creates?
Versions used:
$ yarn -v
3.2.0
$ node -v
v14.16.0
EDIT: So I managed to set up a script that reproduces this issue. It starts a new Nexus server through docker, sets up yarn project, logs in, and tries to add a package.
#!/bin/bash
set -exuo pipefail
UNIQUE_STRING=$(date '+%Y%m%d-%H%M%S')
PROJECT=/tmp/yarn3-auth-repro-$UNIQUE_STRING
# Step 1: Create a new project in /tmp
mkdir -p $PROJECT
cd $_
yarn init -2
echo '
unsafeHttpWhitelist:
- localhost
npmRegistryServer: "http://localhost:8081/repository/npm-all"
npmAlwaysAuth: true
' >>.yarnrc.yml
# Step 2: Set up Nexus
NEXUS_CONTAINER=nexus-repro-container-$UNIQUE_STRING
docker run --rm --detach -p 8081:8081 --name $NEXUS_CONTAINER sonatype/nexus3@sha256:66fe12b1eb3e97bae72eb3c2c4e436499d41ff144cdfd1dcd0718df738304732
function cleanup {
read -p "Stop Nexus docker container $NEXUS_CONTAINER? (y/n) " STOP_CONTAINER
if [[ $STOP_CONTAINER == y* ]]; then
docker stop $NEXUS_CONTAINER
else
echo "You can play around in the project by going to the following directory:"
echo " cd $PROJECT"
echo "To stop the Nexus container, run:"
echo " docker stop $NEXUS_CONTAINER"
fi
}
trap cleanup EXIT
while ! docker logs $NEXUS_CONTAINER | grep 'Started Sonatype Nexus OSS' ; do echo "Waiting for $NEXUS_CONTAINER to start..."; sleep 2; done
printf '\a'
NEXUS_USER=admin
NEXUS_PASS="$(docker exec $NEXUS_CONTAINER cat /nexus-data/admin.password)"
# # Change admin password
# curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/rest/internal/ui/onboarding/change-admin-password' -X PUT --data-raw 'admin123'
# sleep 1
# NEXUS_PASS=admin123
# Disallow anonymous access
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_AnonymousSettings","method":"update","data":[{"enabled":false,"userId":"anonymous","realmName":"NexusAuthorizingRealm"}],"type":"rpc","tid":14}'
# Create npm repo proxy
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_Repository","method":"create","data":[{"attributes":{"npm":{"removeNonCataloged":false,"removeQuarantinedVersions":false},"proxy":{"remoteUrl":"https://registry.npmjs.org","contentMaxAge":1440,"metadataMaxAge":1440},"httpclient":{"blocked":false,"autoBlock":true,"connection":{"useTrustStore":false}},"storage":{"blobStoreName":"default","strictContentTypeValidation":true},"negativeCache":{"enabled":true,"timeToLive":1440},"cleanup":{"policyName":[]}},"name":"npm-proxy","format":"","type":"","url":"","online":true,"routingRuleId":"","authEnabled":false,"httpRequestSettings":false,"recipe":"npm-proxy"}],"type":"rpc","tid":31}'
# Create npm repo group (which is my real setup)
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_Repository","method":"create","data":[{"attributes":{"storage":{"blobStoreName":"default","strictContentTypeValidation":true},"group":{"memberNames":["npm-proxy"]}},"name":"npm-all","format":"","type":"","url":"","online":true,"recipe":"npm-group"}],"type":"rpc","tid":44}'
# Make NPM login work <https://issues.sonatype.org/browse/NEXUS-20170>
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_RealmSettings","method":"update","data":[{"realms":["NexusAuthenticatingRealm","NexusAuthorizingRealm","NpmToken"]}],"type":"rpc","tid":43}'
# Step 3: Login
(echo "$NEXUS_USER"; sleep 2; echo "$NEXUS_PASS") | yarn npm login
yarn npm whoami
# Step 4: Add any random package to make sure auth works (this will fail)
yarn add mkdirp@latest
If you run this script and don't shut down Nexus at the end (you will be asked if you want to), you can cd
to the directory the script created, and manually add the token from $HOME/.yarnrc.yml
to $PROJECT/.yarnrc.yml
, like so:
unsafeHttpWhitelist:
- localhost
npmRegistryServer: "http://localhost:8081/repository/npm-all"
npmAlwaysAuth: true
npmRegistries:
"http://localhost:8081/repository/npm-all":
npmAlwaysAuth: true
npmAuthToken: NpmToken.xxxxxxxxxxxxx
Given the lack of reactions to this issue, I've added a Bash script to the original post which reproduces the error, using a local Nexus running in Docker. Enjoy!
I'm feeling your pain on this one as well. It seems to lose its auth credentials when dealing with non root packages in my monorepo.
I am transitioning our projects to v3, and I have found the failure in authentication to the github package repo.
nodeLinker: pnp
npmScopes:
alienfast:
npmAlwaysAuth: true
npmRegistryServer: "https://npm.pkg.github.com"
yarnPath: .yarn/releases/yarn-3.2.0.cjs
~/p/hello-world-yarn ❯❯❯ yarn npm publish
➤ YN0000: .editorconfig
➤ YN0000: Readme.md
➤ YN0000: bin.js
➤ YN0000: example.gif
➤ YN0000: lib/index.js
➤ YN0000: package.json
➤ YN0000: Package archive published
➤ YN0000: Done in 2s 773ms
~/p/hello-world-yarn ❯❯❯ npm whoami
rosskevin
~/p/hello-world-yarn ❯❯❯ yarn npm whoami
➤ YN0033: No authentication configured for request
➤ YN0000: Failed with errors in 0s 4ms
As you can see, I can publish a private package, but yarn npm whoami
reports no auth whereas npm whoami
reports my rosskevin
user.
I investigated this path because previous packages published on https://npm.pkg.github.com/ are reporting unavailable with misleading messages such as:
~/p/hello-world-yarn ❯❯❯ yarn add @alienfast/eslint-config
➤ YN0000: ┌ Resolution step
➤ YN0033: │ eslint-config-prettier@npm:^8.5.0: No authentication configured for request
➤ YN0000: └ Completed in 3s 875ms
➤ YN0000: Failed with errors in 3s 880ms
whereas npm
works fine:
~/p/hello-world-yarn ❯❯❯ npm add @alienfast/eslint-config ✘ 1
added 195 packages, and audited 196 packages in 13s
64 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
TL;DR yarn 3 authentication is not working with older github artifacts, and yarn npm whoami
is misreporting No authentication configured for request
even though yarn npm publish
successfully publishes to the github repository!
~/p/hello-world-yarn ❯❯❯ yarn npm whoami --scope alienfast ✘ 1
➤ YN0000: rosskevin
➤ YN0000: Done in 0s 348ms
(as an aside, I tried publishing a current @alienfast/eslint-config with yarn npm publish
which is also successful but no yarn command can pull from that repo, even though npm can).
SO ... WHY can I publish but not install from the same config?
Bizarrely, yarn config
shows opposite of what I have scoped:
➤ YN0000: npmRegistries Map(1) { 'https://npm.pkg.github.com' => Map(3) { 'npmAlwaysAuth' => false, 'npmAuthIdent' => null, 'npmAuthToken' => '********' } }
➤ YN0000: npmRegistryServer 'https://registry.yarnpkg.com'
➤ YN0000: npmScopes Map(1) { 'alienfast' => Map(6) { 'npmAlwaysAuth' => true, 'npmAuthIdent' => null, 'npmAuthToken' => null, 'npmAuditRegistry' => null, 'npmPublishRegistry' => null, 'npmRegistryServer' => 'https://npm.pkg.github.com' } }
➤
npmAlwaysAuth
is clearly set to true under the alienfast
scope IN the project's yarnrc.yml
.
Ok, so this seems to look like something attempted to be solved by the deep merge PR from #2106 which was rejected due to major breaking changes?
This is a bug, confirmed. The only way I can (so far) get yarn config
to appear properly and yarn add
to work properly is to include my token in the project's yarnrc.yml
(huge security hole) and specify my auth with BOTH the npmRegistries
key AND npmScopes
.
e.g.
nodeLinker: pnp
npmRegistries:
"https://npm.pkg.github.com":
npmAuthToken: xxxxx
npmAlwaysAuth: true
npmScopes:
alienfast:
npmAlwaysAuth: true
npmRegistryServer: "https://npm.pkg.github.com"
yarnPath: .yarn/releases/yarn-3.2.0.cjs
^^^ this works as a project yarnrc.yml
but is unusable from a security perspective.
For the record, I've been trying to get this working in Sherlock, but can't really figure out how to pass username/password to yarn npm login
. The shell trick (echo "$NEXUS_USER"; sleep 2; echo "$NEXUS_PASS") | yarn npm login
which I use above doesn't seem to work when running in Sherlock.
I found a solution in another issue. It is required to specify the token inside npmRegistries
field. So your ~/.yarnrc.yml
should look like:
npmRegistries:
"https://my-registry-url":
npmAuthToken: ...
npmScopes:
myscope:
npmAuthToken: ...
Not sure if both are required, but the first one certainly is.
@alamothe this issue is called "yarn 3 ignores npmAuthTokens in ~/.yarnrc.yml" :) Running yarn npm login
adds the token to ~/.yarnrc.yml
, so that's not a problem; the problem is that yarn doesn't use the token from ~/.yarnrc.yml´ when I run
yarn install`.
Don't know if adding npmScopes
changes anything, but we don't use scopes so that's an academic question. We use a Nexus server which both acts as proxy for the public NPM repo and serves our own private packages, no scopes involved, auth always required.
@gustafc Interesting. It did ignore my token in ~/.yarnrc.yml
if I only used npmScopes
, but once I added npmRegistries
too, it started to use it.
I definitely agree that there is a bug, I just wanted to help with a potential workaround until it is solved.
It doesn't ignore the token, it just doesn't support configuration merging. In other words, the npmRegistries
you declare in your project yarnrc overrides the one in the global one (same for npmScopes
).
That's something we'll fix in the next major, since it's a breaking change, but it's quite tricky to implement in a satisfying way, so it hasn't been started yet.
@arcanis I see. I eagerly await 4.0 then!
Wow, so the reason it works for people who use scopes is that it can take npmScopes
from project .yarnrc.yml
and npmRegistries
from home dir .yarnrc.yml
. This way it is possible to separate scopes/registries configuration from tokens.
I don't see a workaround for people who don't use scopes.
@arcanis IMHO the only practical reason for separating the config is auth tokens, so perhaps that could simplify the solution i.e. it doesn't really need to be general.
It doesn't ignore the token, it just doesn't support configuration merging. In other words, the
npmRegistries
you declare in your project yarnrc overrides the one in the global one (same fornpmScopes
).That's something we'll fix in the next major, since it's a breaking change, but it's quite tricky to implement in a satisfying way, so it hasn't been started yet.
@arcanis Is there an issue for the configuration merging that can be tracked? Or should I create one? It is quite a roadblock for us and we are very keen to get this feature in.
it doesn't work for me with scoped packages on yarn 3.2.3 @__@
having a .yarnrc.yml like this:
yarnPath: .yarn/releases/yarn-3.2.3.cjs
nodeLinker: node-modules
npmRegistries:
"https://npm.pkg.github.com":
npmAlwaysAuth: true
npmScopes:
pkgscope:
npmAlwaysAuth: true
npmRegistryServer: "https://npm.pkg.github.com"
I still get ➤ YN0041: │ @pkgscope/package-name@npm:^0.1.0: Invalid authentication (as an anonymous user)
on yarn add
@arcanis Can we expect this change for v3 or will this just be included in v4? Because it isn't working in 3.3.1 for scoped packages.
project's .yarnrc.yml:
npmScopes:
myscope:
npmRegistryServer: "https://npm.pkg.github.com"
npmAlwaysAuth: true
Login via yarn npm login -s myscope
results in ~/.yarnrc.yml:
nodeLinker: node-modules
npmScopes:
myscope:
npmAuthToken: my-secret-auth-token
But installing a package from "myscope" in the exampke above results in "YN0041: Invalid authentication (as an anonymous user)". As a workaround, it is working with the following ~/.yarnrc.yml (no changes to the project's .yarnrc.yml:
nodeLinker: node-modules
npmRegistries:
https://npm.pkg.github.com:
npmAuthToken: my-secret-auth-token
But then I have to do manual work, because yarn npm login
doesn't support logging in into a specific repository via parameter.
It'll not be backported to 3.x considering it's a breaking change, and very difficult to cherry pick anyway. If you need it, please use the RCs.
Can someone please summarize what has to go in project/.yarnrc.yml
and ~/.yarnrc.yml
in version 3.x to publish without including the authentication tokens as part of the project?
I tried the following and it did not work:
project/.yarnrc.yml:
npmScopes:
public:
npmRegistryServer: "https://registry.npmjs.org"
npmAlwaysAuth: true
~/.yarnrc.yml
npmRegistries:
"https://registry.npmjs.org":
npmAuthToken: xxx
I still get YN0033: No authentication configured for request
when I run yarn npm publish --access=public
I figured it out. I needed to add npmPublishRegistry: "https://registry.npmjs.org"
to ~/.yarnrc.yml
.
I had the same problem and solved it like this.
to ~/.yarnrc.yml
yarnPath: .yarn/releases/yarn-3.4.1.cjs
nodeLinker: node-modules
npmScopes:
your-company:
npmAlwaysAuth: true
npmRegistryServer: "https://nexus.your-company.io/repository/your-npm/"
npmAuthIdent: "username:password"
https://yarnpkg.com/configuration/yarnrc#npmAuthIdent
well this is problematic and commenters don't quite grok what's happening so hopefully this explanation clears it up:
if you have a npmScopes: your-company: block in your ~/.yarnrc.yml, any keys there will be completely ignored (not merged) if there's also a npmScopes: your-company: block on your project's .yarnrc.yml. Therefore, whatever you try to use inside the npmScopes: your-company: block in ~/.yarnrc.yml won't work at all. It will only work if there's no npmScopes: your-company: block in your project's .yarnrc.yml
yarn 4 fixes this no merging issue and will apply the project's .yarnrc.yml over ~/.yarnrc.yml but keeping the values of keys that are specified in ~/.yarnrc.yml but not in the project's .yarnrc.yml. so if you have a npmAlwaysAuth: true for the your-company scope on ~/.yarnrc.yml and not in the project's .yarnrc.yml where you have other keys for the your-company scope, on yarn 4 it will respect the npmAlwaysAuth from ~/.yarnrc.yml, while on yarn 3 it will be false as it is the default and it ignores the your-company scope block from your home's .yarnrc.yml because no merging.
hope this helps anyone and hope yarn 4 comes out soon officially.
For anyone using v3 still and looking for a solution, you can use env variables as a workaround, eg:
npmAuthToken: '${SOME_TOKEN:-}' // :- is added as a fallback to prevent a missing token from causing issues when the variable isn't necessary
and then use it:
SOME_TOKEN=<your token here> yarn npm publish
and of course, since it's an env variable, you can use other ways of loading it.
I am on yarn 3.5.1 have the local .yarnrc.yml defined like this:
`nodeLinker: node-modules
plugins:
- path: .yarn/plugins/plugin-dedupe-on-install.cjs spec: "https://raw.githubusercontent.com/ambar/yarn-plugin-dedupe-on-install/main/index.js"
- path: .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs spec: "@yarnpkg/plugin-workspace-tools"
npmRegistries: "REGISTRY URL": npmAuthToken: "TOKEN" npmAlwaysAuth: true
npmScopes: platform-ui: npmRegistryServer: REGISTRY URL npmAlwaysAuth: true npmAuthToken: 'TOKEN'
yarnPath: .yarn/releases/yarn-3.5.1.cjs`
And i am still getting this error:
`➤ YN0027: @platform-ui/design-system@unknown can't be resolved to a satisfying range ➤ YN0041: Invalid authentication (as an unknown user)
➤ Errors happened when preparing the environment required to run this command.`
@asingh072318 I had the same problem just now. Upgrading to Yarn v4 solved it.
Ran into this issue while using a Sonatype Nexus Repository and the fix was using new token formatting.
npmAuthToken: https://nexus.intern.***.***.***/repository/npm-group/:authToken=XXXX-XXXXX-XXX
To anyone running into this problem, the solution to the problem I had was:
- Using yarn 4 (since yarn 4 tries to merge configs from
~/.yarnrc.yml
and$PROJECT/.yarnrc.yml
) - Logging in with
yarn npm login --always-auth
(since yarn 4 isn't that good at merging configs, this addsnpmAlwaysAuth
to thenpmAuthToken["https://nexus.example.com/repository/npm-all"]
entry in~/.yarnrc.yml
which would otherwise default to false and override the top-levelnpmAlwaysAuth
in$PROJECT/.yarnrc.yml
)
I'd still argue it's a bug that project-specific npmAlwaysAuth
doesn't influence the behaviour of yarn 4, and thus 👉 #6097