nx icon indicating copy to clipboard operation
nx copied to clipboard

Adding a package.json to a lib will prevent NX looking for .npmrc files in the root.

Open grumpyoldman-io opened this issue 1 year ago • 1 comments

Current Behavior

When running a release publish to a private Github package repository NX is unable to add the correct authentication token via .npmrc from the repo's root .npmrc when that lib (in a subfolder like ./libs/my-test-lib) has a package.json.

Expected Behavior

When running a release publish and my libs contain a package.json but no .npmrc (as it is in the root of the repo) it should use the roots' .npmrc file for publishing

GitHub Repo

https://github.com/nrwl/nx-examples

Steps to Reproduce

Since this involves private settings I sadly can't create a repo that produces this but these are the steps:

  1. create empty multi package node repo with NX
  2. create publishable library (example name: @my-test-scope/my-test-lib)
  3. add .npmrc to the root with something like (make sure this is not present in your home folder, as that is respected):
    @my-test-scope:registry=https://npm.pkg.github.com
    //npm.pkg.github.com/:_authToken=${MY_AUTH_TOKEN}
    
  4. run nx release publish

You should be confronted with auth issues.

as soon as you add a copy of your .npmrc to your lib folder, it will work

Nx Report

Node   : 20.10.0
OS     : darwin-arm64
yarn   : 1.22.21

nx                 : 18.0.3
@nx/js             : 18.0.3
@nx/jest           : 18.0.3
@nx/linter         : 18.0.3
@nx/eslint         : 18.0.3
@nx/workspace      : 18.0.3
@nx/devkit         : 18.0.3
@nx/eslint-plugin  : 18.0.3
@nx/node           : 18.0.3
@nrwl/tao          : 18.0.3
typescript         : 5.3.3

Failure Logs

No response

Package Manager Version

No response

Operating System

  • [X] macOS
  • [ ] Linux
  • [ ] Windows
  • [ ] Other (Please specify)

Additional Information

I've debugged the issue after I found this comment:

https://github.com/nrwl/nx/pull/16652/files#diff-5436947051cc756ff8c6940d47a4d78f54570bbaaa5947ab6c007639d5bdfd17R119-R120

grumpyoldman-io avatar Feb 13 '24 10:02 grumpyoldman-io

I had a similar issue, but for me, I passed the GITHUB_TOKEN with --with-env-vars flag

npx nx-cloud start-ci-run --distribute-on="8 linux-medium-js" --stop-agents-after="e2e"
          --with-env-vars="GITHUB_TOKEN"

sarea avatar Feb 14 '24 13:02 sarea

Yeah happening in our project as well. looking to create an executor to copy the npmrc into the project folder until this is sorted

makirby avatar Feb 22 '24 15:02 makirby

Having the same issue here. Any updates on this?

zelladays avatar Feb 22 '24 17:02 zelladays

Running into the same problems unfortunately as above... 😢

keenanzucker-tipalti avatar Feb 23 '24 08:02 keenanzucker-tipalti

^ Same. Any update much appreciated.

ruairidhflint avatar Feb 23 '24 10:02 ruairidhflint

Experiencing the same problem. Hopefully, it will be resolved soon

AnastasiaGridchina avatar Feb 23 '24 10:02 AnastasiaGridchina

Hi Folks, I believe this is a limitation on the npm side, ultimately nx release publish is running npm publish behind the scenes, and I don't think it is respecting any kind of inheritance up multiple levels like that.

Please could one of the people on this thread try publishing their setup using npm publish directly in on of their project directories to confirm?

E.g.

.npmrc
package.json
# ... other root files and folders
packages/
  packageA/
    package.json
    # ... other project files and folders

If you run npm publish directly within the packages/packageA directory, does it respect the root .npmrc?

JamesHenry avatar Feb 23 '24 14:02 JamesHenry

Hey @JamesHenry, thanks for taking a look into this.

I see, npm publish does not respect the root .npmrc, that is a shame.

My solution (so I can continue publishing after switching to nx release) has been to patch the release-publish executor to copy the .npmrc over to the package root. (terrible idea but works for now)

Thinking longer term, I dont really fancy maintaining a .npmrc in every single package rootwhen they will all be carbon copies of the root one. I wonder if there is something we can do to ease this when using nx release?

makirby avatar Feb 23 '24 15:02 makirby

Thanks for confirming this is an npm limitation @makirby, we are discussing ways we can work around this - they will likely need to be opt in as copying the .npmrc by default is a potentially risky thing to do

JamesHenry avatar Mar 11 '24 14:03 JamesHenry

This is the PR that is addressing this issue: https://github.com/nrwl/nx/pull/22304

JamesHenry avatar Mar 15 '24 09:03 JamesHenry

@JamesHenry Thanks for picking this up. I wouldn't mind adding something like a flag or config to opt in to this behavior, and seeing the comments and solutions mentioned in this thread I think a lot of people have made "worse" workarounds. I do hope that the config will exist on the package level, not the root. Something along the lines of "useNpmrcFromRoot" (terrible naming, but just to make a point) would make it possible to reuse it, if it was possible to extend or compose the root file that would be a cherry on top for me.

grumpyoldman-io avatar Mar 18 '24 07:03 grumpyoldman-io

@grumpyoldman-io allow me to invert the thought process for the purposes of discussion - why would you put properties in a root config that don’t apply in the majority of circumstances?

If you have project specific values that are needed to a minority of projects, adding project level .npmrc files for that purpose (which would be respected) seems reasonable to me.

That is how almost all other tools work, always taking the root config into account and then overriding with project level config where necessary.

It’s also worth noting that not all the commands with the executor have actually been performing consistently because of these npm limitations so that PR genuinely is a bug fix as well unlocking new capabilities when it comes to auth properties in the root

JamesHenry avatar Mar 18 '24 08:03 JamesHenry

why would you put properties in a root config that don’t apply in the majority of circumstances?

My thought process was based on the notion of making this behavior explicit, and opt-in per package so you can cherry pick when to apply this as to my knowledge npmrc has no way of extending / overriding. Your following point though:

That is how almost all other tools work, always taking the root config into account and then overriding with project level config where necessary.

Makes a lot of sense to me, and if the override per project also allows you to change root settings this would work perfectly for all cases I can come up with. I was just not aware that this would be a possibility for .npmrc. A lot of the other tools in the repo have explicit "extends" or "import base" like syntax though, so Im curious how this would look like in practice.

grumpyoldman-io avatar Mar 18 '24 09:03 grumpyoldman-io

Yes I agree, I think npm config files are lacking explicit mechanisms for inheritance.

One nuance here is if publishing from dist, the .npmrc at the project level needs to be copied to that location as part of the build step…

There isn’t really much we can do to automate this I don’t think but we can document it in @fahslaj PR

JamesHenry avatar Mar 18 '24 10:03 JamesHenry

AFAIK npm is not very strict about what it contains, just thinking out loud, perhaps introducing a custom setting (either in .npmrc or in nx.json) to do some "just in time" .npmrc creation/removal could potentially work?

To me this might just fall within the scope of monorepo tooling, but having a comment in the docs telling people to manually create the .npmrc per package will also do the trick with a whole lot less logic.

grumpyoldman-io avatar Mar 20 '24 08:03 grumpyoldman-io

Ok update on this folks, we are going to have a call with the npm team next week on this to make sure we implement the solution that will work best/most intuitively for you all.

We still have some open questions about how npm CLI is intended to work in certain scenarios (e.g. it seems that sadly I was wrong about project level .npmrc files being respected if you run the command from the workspace root pointing at the project directory) and this will be our chance to address all of them before closing this issue.

I'll keep you posted!

JamesHenry avatar Mar 21 '24 08:03 JamesHenry

Hey everyone! This has been fixed with this PR - https://github.com/nrwl/nx/pull/22304

Now, nx release will use the root .npmrc when publishing. This includes support for scoped registries and auth within .npmrc. If you need to override the registry at the project-level (instead of by npm scope), then you can set the registry and tag options in project configuration. See this doc for details - https://nx.dev/recipes/nx-release/configure-custom-registries

fahslaj avatar Apr 02 '24 20:04 fahslaj

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

github-actions[bot] avatar May 03 '24 00:05 github-actions[bot]