`nx release` updating lock file un-syncs lock file with package.json

[m-thalmann]

Current Behavior

When executing the nx release command the package-lock.json file is updated (which is expected). But it also removes peer-dependencies which are actually in use. Therefore subsequent npm ci calls fail, because the package.json and the package-lock.json file are not in sync anymore (executing npm i adds the dependencies back to the lock file).

In my case this refers to the following transitive (peer-)dependencies of @nx/js:

• @types/node
• typescript
• undici-types

Expected Behavior

The nx release command updates the package-lock file, but only updates the versions of the newly release packages and does not remove any used (peer-)dependencies

GitHub Repo

https://github.com/m-thalmann/codestyle-packages/

Steps to Reproduce

1. Create a Nx Package-Based project (pure JS, no Typescript)
2. Create a package and commit the changes
3. Setup Nx Release
   • I used the following config:

     {
       "release": {
         "projects": ["*"],
         "projectsRelationship": "independent",
         "version": {
           "conventionalCommits": true
         },
         "git": {
           "commitMessage": "build(release): create new versions of packages"
         },
         "changelog": {
           "projectChangelogs": {
             "renderOptions": {
               "authors": false
              }
           }
         }
       }
     }
     

4. Run nx release --skip-publish --first-release (or omit the skip-publish, does not matter) (its also not dependent on the first release flag)
5. Inspect the create commit: The package-lock should be updated with the above mentioned dependencies removed
6. Run npm ci -> this will fail

Nx Report

Node   : 20.11.0
OS     : win32-x64
npm    : 10.2.4

nx (global)    : 18.0.3
nx             : 18.0.5
@nx/js         : 18.0.5
@nx/workspace  : 18.0.5
@nx/devkit     : 18.0.5
@nrwl/tao      : 18.0.5
typescript     : 5.3.3

Failure Logs

No response

Package Manager Version

No response

Operating System

• [ ] macOS
• [X] Linux
• [X] Windows
• [ ] Other (Please specify)

Additional Information

This issue occurs on both our local windows developer machines and the linux build servers.

I tried to find the root cause but I was not able to reproduce this issue when running the npm install --package-lock-only command (which is used by the update lock file function in our case). I've also tried executing it from a custom JS script but this did also not reproduce the above mentioned issue.

Maybe it is something in the packages/js/src/generators/release-version/utils/update-lock-file.ts file but I have no idea how to start.

[lancempoe]

I am getting a similar error. I'll note that it also fails when setting skipLockFileUpdate as noted in the error logs shown below. We are blocked now :( Maybe there is a version of 18 that I can use that doesn't have this error?

I'll also note that there is no documentation for skipLockFileUpdate in the readme so adding something there might be nice.

NX Updating npm lock file npm ERR! Cannot read properties of null (reading 'package') npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-03-02T06_23_02_573Z-debug-0.log NX Error updating lock file with command 'npm install --package-lock-only' Verify that 'npm install --package-lock-only' succeeds when run from the workspace root. To configure a string of arguments to be passed to this command, set the 'release.version.generatorOptions.installArgs' property in nx.json. To ignore install lifecycle scripts, set 'release.version.generatorOptions.installIgnoreScripts' to true in nx.json. To disable this step entirely, set 'release.version.skipLockFileUpdate' to true in nx.json.

Here are my settings:

"release": { "projectsRelationship": "independent", "version": { "conventionalCommits": true }, "changelog": { "projectChangelogs": true } } },

Relavent package versions:

"nx": "^18.0.6"
"@nrwl/cli": "^15.9.7",
"@nx/eslint-plugin": "^18.0.6",
"@nx/devkit": "^18.0.6",
"@nx/jest": "^18.0.6",
"@nx/js": "^18.0.6",
"@nx/node": "^18.0.6",

The final note is that it changes our internal dependencies from * to a direct number and that breaks the lock file

[ryan-mcginty-alation]

I am also experiencing a similar issue. npm ci fails after publish run due to peer dependency issues after release command updates the lock file incorrectly.

Checking to see if https://github.com/s1seven/schema-tools/issues/253 fixes it

[ryan-mcginty-alation]

Still not working for me, below is the error on CI/CD. It requires re-running npm i to regenerate the lock file to fix it. Using Nx 18.0.8

npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @vitejs/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/vite
npm WARN   dev vite@"5.0.10" from the root project
npm WARN   15 more (@joshwooding/vite-plugin-react-docgen-typescript, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer vite@"^4.1.0-beta.0" from @vitejs/[email protected]
npm WARN node_modules/@storybook/react-vite/node_modules/@vitejs/plugin-react
npm WARN   @vitejs/plugin-react@"^3.0.1" from @storybook/[email protected]
npm WARN   node_modules/@storybook/react-vite
npm WARN 
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/vite
npm WARN   peer vite@"^4.1.0-beta.0" from @vitejs/[email protected]
npm WARN   node_modules/@storybook/react-vite/node_modules/@vitejs/plugin-react
npm WARN     @vitejs/plugin-react@"^3.0.1" from @storybook/[email protected]
npm WARN     node_modules/@storybook/react-vite
npm ERR! code EUSAGE
npm ERR! 
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR! 
npm ERR! Missing: [email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/swc-win[32]([email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @next/[email protected] from lock file
npm ERR! Missing: @swc/[email protected] from lock file
npm ERR! Missing: [email protected] from lock file
npm ERR! Missing: [email protected] from lock file
npm ERR! Missing: [email protected] from lock file
npm ERR! Missing: @storybook/[email protected] from lock file
npm ERR! Missing: @storybook/[email protected] from lock file
npm ERR! Missing: [email protected] from lock file
npm ERR! 
npm ERR! Clean install a project
npm ERR! 
npm ERR! Usage:
npm ERR! npm ci
npm ERR! 
npm ERR! Options:
npm ERR! [--install-strategy <hoisted|nested|shallow|linked>] [--legacy-bundling]
npm ERR! [--global-style] [--omit <dev|optional|peer> [--omit <dev|optional|peer> ...]]
npm ERR! [--strict-peer-deps] [--foreground-scripts] [--ignore-scripts] [--no-audit]
npm ERR! [--no-bin-links] [--no-fund] [--dry-run]
npm ERR! [-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
npm ERR! [-ws|--workspaces] [--include-workspace-root] [--install-links]
npm ERR! 
npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
npm ERR! 
npm ERR! Run "npm help ci" for more info

[ryan-mcginty-alation]

Finally figured out a workaround in programmatic api:

const {workspaceVersion, projectsVersionData} = await releaseVersion({
    specifier: options.version,
    dryRun: options.dryRun,
    verbose: options.verbose,
    gitCommit: false, // Commit will be created in the releaseChangelog step
    gitTag: false, // Tag will be created in the releaseChangelog step
  });

  // workaround for https://github.com/nrwl/nx/issues/22066
  console.log('un-stage and checkout package-lock.json');
  execSync('git restore --staged package-lock.json', {stdio: 'inherit'});
  execSync(`git checkout package-lock.json`, {stdio: 'inherit'});

  await releaseChangelog({
    gitCommitMessage: `chore: ${options.issueKey}: release version {version}`,
    versionData: projectsVersionData,
    version: workspaceVersion,
    dryRun: options.dryRun,
    verbose: options.verbose,
  });

[ThePlenkov]

Please provide the option to disable updating package-lock.json with release. It's basically breaking things. After this command next npm install removes dozens of depencies from my project and it stops working locally. The only way to fix - is to remove node_modules and package-lock.json and reinstall. This is very inconvenient..

[ThePlenkov]

OK just found it:

  "release": {
    "version": {
      "generatorOptions": {
        "skipLockFileUpdate": true
      }
    }
  }

made a trick in nx.json. Now it's good

[bentinata]

@ThePlenkov Wouldn't this make lockfile unsync with package.json since nx release is updating version in package.json, but not in the package-lock.json?

[ThePlenkov]

Absolutely correct, that's what I fix with usage of workspaces + npm install package lock only in post install script. Works like a charm

[elliewintram]

I was also experiencing this issue whereby the nx release command was removing peer deps from the package-lock.json, resulting in npm ci failing on subsequent builds due to the package.json and package-lock.json being out of sync (using node v20, npm v10).

For me it seemed to be caused by the fact that nx release calls npm install with the legacy-peer-deps set to true, seen in the source code here.

Following the docs for npm ci, I added legacy-peer-deps=true to the .npmrc file to ensure peer deps are not installed on any npm install command, which is the behaviour of older npm versions.

NOTE: If you create your package-lock.json file by running npm install with flags that can affect the shape of your dependency tree, such as --legacy-peer-deps or --install-links, you must provide the same flags to npm ci or you are likely to encounter errors. An easy way to do this is to run, for example, npm config set legacy-peer-deps=true --location=project and commit the .npmrc file to your repo.

This removed the peer deps in my package-lock.json and resulted in no further changes to the package-lock.json on nx release.

Question for the nx team - is process.env.npm_config_legacy_peer_deps ??= 'true'; going to be removed in a future version based on the comment in the source code?

[arichard-info]

Hello everyone,

Like you, I have the package-lock.json problem, I'm also looking for solutions. But something struck me in the discussions: Is it really an npm install that is performed by nx release to re-synchronise the package-lock.json?

https://github.com/nrwl/nx/blob/master/packages/nx/src/utils/package-manager.ts#L160

Is this deliberate? It's likely to update dependencies that aren't affected by the release, isn't it? I was right to look into this because I thought that the package-lock.json was updated in a much more meticulous way, with only the versions affected.

If that's the case, it's highly inadvisable to run this command in a CI, isn't it?