0x80
PNPM prune fails to read 'specifiers'
Hi thanks for isolate-package. I'm interested in this project, I think it could help me build docker images from my pnpm monorepo. I followed the prerequisites as best I could, but I ran into an error when I tried to isolate one of my packages.
npx isolate-package isolate ✔ ▼
info Using isolate-package version 1.18.0
Error: Failed to generate lockfile: Cannot read properties of undefined (reading 'specifiers')
at generatePnpmLockfile (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/lib/lockfile/helpers/generate-pnpm-lockfile.ts:175:11)
at processLockfile (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/lib/lockfile/process-lockfile.ts:69:7)
at isolate (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/isolate.ts:175:29)
at run (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/isolate-bin.ts:9:3)
I'm not sure what the 'specifiers' thing is. I searched isolate-package source code for that string and it looks like it's maybe the package version specifier in package.json?
example: "@futurenet/scout": "workspace:^",
Curiously, isolate-package did generate a bunch of output in the isolate directory. I'm not sure if it completed it's job, though, since I haven't seen what a successful run of isolate looks like.
Not sure if it's something I did wrong with my project configuration but I figured I'd document what I came across. I think my next move is to double check that my package build process is correct. Right now, all of my packages have a build step. I read a bunch of articles and docs pages about isolate-package and I'm learning about internal packages for the first time. Maybe I will try using internal packages instead.
Another thing I'm suspicious of, maybe it's causing conflicts, is how Temporal loads it's workflows. My project integrates with Temporal and Temporal insists on using webpack to load workflow files. Normally I wouldn't use webpack at all, I'd use tsc instead. I'm not sure how that ties into everything but maybe what I'll do is see if I can re-create this error on a simpler project with fewer dependencies.
Edit 1
I made a simplified git repo and successfully used isolate-package to isolate the taco package. taco package depends on meat package and isolate-package ran without error. This points to the issue being with my code.
@insanity54 I have just released an update that improves error logging by re-throwing the original error so you should get a full stack trace. Hope it helps.
If you can't find the problem in your codebase, and you are willing to give me access, then I could take a look.
@insanity54 I was getting the same error and created a small app to try to reproduce without success; it worked just fine! I then tried to replicate with the same scope of dependencies on a fresh directory and it still worked as expected... Hmm, very strange! The following does not identify the root cause, but helped me get unstuck. I hope it helps you.
Error output with debug logging
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> npx isolate
info Using isolate-package version 1.19.0
debug Using buildDirName from config: .
debug Workspace root resolved to C:\Users\thiles\gitrepo\nodered
debug Isolate target package (root)\app\sfUserIntegration
debug Isolate output directory (root)\app\sfUserIntegration\isolate
debug Cleaned the existing isolate output directory
debug Detected package manager pnpm 9.5.0+sha512.140036830124618d624a2187b50d04289d5a087f326c9edfc0ccd733d76c4f52c3a313d4fc148794a2a9d81553016004e6742e8cf850670268a7387fc220c903
debug Use PNPM pack instead of NPM pack
debug Detected pnpm packages globs: [ 'app/sfUserIntegration' ]
debug Registering package app\sfUserIntegration
debug Packed (temp)/sfUserIntegration-1.0.0.tgz
debug Generating PNPM lockfile...
debug Relevant importer ids: [ 'app/sfUserIntegration' ]
debug Pruning the lockfile
error Failed to generate lockfile: Cannot read properties of undefined (reading 'specifiers')
TypeError: Cannot read properties of undefined (reading 'specifiers')
at pruneLockfile (C:\Users\thiles\gitrepo\nodered\node_modules\.pnpm\@[email protected]\node_modules\@pnpm\prune-lockfile\src\index.ts:56:56)
at generatePnpmLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/lib/lockfile/helpers/generate-pnpm-lockfile.ts:145:15)
at processLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/lib/lockfile/process-lockfile.ts:69:7)
at isolate (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/isolate.ts:175:29)
at run (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/isolate-bin.ts:9:3)
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> npx isolate
info Using isolate-package version 1.19.0
debug Using buildDirName from config: .
debug Workspace root resolved to C:\Users\thiles\gitrepo\nodered
debug Isolate target package (root)\app\sfUserIntegration
debug Isolate output directory (root)\app\sfUserIntegration\isolate
debug Cleaned the existing isolate output directory
debug Detected package manager pnpm 9.5.0+sha512.140036830124618d624a2187b50d04289d5a087f326c9edfc0ccd733d76c4f52c3a313d4fc148794a2a9d81553016004e6742e8cf850670268a7387fc220c903
debug Use PNPM pack instead of NPM pack
debug Detected pnpm packages globs: [ 'app/sfUserIntegration' ]
debug Registering package app\sfUserIntegration
debug Packed (temp)/sfUserIntegration-1.0.0.tgz
debug Generating PNPM lockfile...
debug Relevant importer ids: [ 'app/sfUserIntegration' ]
debug Pruning the lockfile
error Failed to generate lockfile: Cannot read properties of undefined (reading 'specifiers')
TypeError: Cannot read properties of undefined (reading 'specifiers')
at pruneLockfile (C:\Users\thiles\gitrepo\nodered\node_modules\.pnpm\@[email protected]\node_modules\@pnpm\prune-lockfile\src\index.ts:56:56)
at generatePnpmLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/lib/lockfile/helpers/generate-pnpm-lockfile.ts:145:15)
at processLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/lib/lockfile/process-lockfile.ts:69:7)
at isolate (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/isolate.ts:175:29)
at run (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/[email protected]/node_modules/isolate-package/src/isolate-bin.ts:9:3)
Cleanup pnpm-lock files
The error message related to pruning the lock file gave me the hunch to remove all existing pnpm-lock.yaml files as follows:
- removed all
pnpm-lock.yamlfiles in all subdirectories from the root monorepo - removed
node_modulesrecursively -
pnpm install -
npx isolate
It now produced the expected isolate output without the error! Conclusion/assumption: An old lock file was being processed
Output after Cleanup with debug logging
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> pnpm install
Scope: all 2 workspace projects
WARN 7 deprecated subdependencies found: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
../.. | +631 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
../.. | Progress: resolved 631, reused 631, downloaded 0, added 631, done
../../node_modules/.pnpm/[email protected][email protected]/node_modules/bcrypt: Running install script, done in 593ms
../../node_modules/.pnpm/[email protected][email protected]/node_modules/bcrypt: Running install script, done in 604ms
dependencies:
+ @thiles/node-red-contrib-amqp 2.4.4
+ do-red 0.3.5 (1.0.1 is available)
+ moment 2.30.1
+ node-red 3.1.11 (4.0.2 is available)
+ node-red-contrib-activedirectory 0.3.2
+ node-red-contrib-better-ldap-servicely 0.1.7
+ node-red-contrib-countula 1.1.9
+ node-red-contrib-cron-plus 1.5.7 (2.1.0 is available)
+ node-red-contrib-flogger 1.2.2
+ node-red-contrib-flowtest 0.1.2
+ node-red-contrib-moment 4.0.0 (5.0.0 is available)
+ node-red-contrib-mongodb3 2.0.1
+ node-red-contrib-oauth2 4.0.3 (6.2.1 is available)
+ node-red-contrib-objectid 1.1.0
+ node-red-contrib-oracledb-2 0.3.3
+ node-red-contrib-powershell 0.0.3 (0.0.5 is available)
+ node-red-contrib-reusable-flows 1.0.0
+ node-red-contrib-salesforce-fix 0.8.5
+ node-red-contrib-semaphore 1.0.1
+ node-red-node-daemon 0.4.0 (0.6.0 is available)
+ oracledb 5.5.0 (6.6.0 is available)
+ xml2js 0.4.23 (0.6.2 is available)
Done in 9.3s
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> npx isolate
info Using isolate-package version 1.19.0
debug Using buildDirName from config: .
debug Workspace root resolved to C:\Users\thiles\gitrepo\nodered
debug Isolate target package (root)\app\sfUserIntegration
debug Isolate output directory (root)\app\sfUserIntegration\isolate
debug Detected package manager pnpm 9.5.0+sha512.140036830124618d624a2187b50d04289d5a087f326c9edfc0ccd733d76c4f52c3a313d4fc148794a2a9d81553016004e6742e8cf850670268a7387fc220c903
debug Use PNPM pack instead of NPM pack
debug Detected pnpm packages globs: [ 'app/sfUserIntegration' ]
debug Registering package app\sfUserIntegration
debug Packed (temp)/sfUserIntegration-1.0.0.tgz
debug Generating PNPM lockfile...
debug Relevant importer ids: [ 'app/sfUserIntegration' ]
debug Setting target package importer on root
debug Pruning the lockfile
debug Created lockfile at C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration\isolate\pnpm-lock.yaml
debug Copied .npmrc file to the isolate output
debug Deleting temp directory (root)\app\sfUserIntegration\isolate\__tmp
info Isolate completed at C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration\isolate
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration>
isolate.config.json
{
"logLevel": "debug",
"buildDirName": "."
}
app\sfUserIntegration\package.json with Catalog: scheme pnpm version specifiers
{
"description": "Nodered - Flows for provisioning Georgian Users; eg. sensing Salesforce Contact's Admission Group needs",
"name": "sfUserIntegration",
"version": "1.0.0",
"scripts": {
"prestart": "nvs auto",
"start": "node --max-old-space-size=8196 --env-file .env node_modules/node-red/red.js --userDir=. --port=1884"
},
"files": [
"flows.json",
"flows_cred.json",
"settings.js",
"Microsoft.PowerShell_profile.ps1",
"pnpm-lock.yaml"
],
"dependencies": {
"@thiles/node-red-contrib-amqp": "catalog:messaging",
"do-red": "~0.3.5",
"moment": "catalog:essential",
"node-red": "catalog:essential",
"node-red-contrib-activedirectory": "^0.3.2",
"node-red-contrib-better-ldap-servicely": "~0.1.7",
"node-red-contrib-countula": "catalog:essential",
"node-red-contrib-cron-plus": "catalog:essential",
"node-red-contrib-flogger": "catalog:essential",
"node-red-contrib-flowtest": "~0.1.2",
"node-red-contrib-moment": "catalog:essential",
"node-red-contrib-mongodb3": "catalog:db",
"node-red-contrib-oauth2": "catalog:api",
"node-red-contrib-objectid": "~1.1.0",
"node-red-contrib-oracledb-2": "catalog:db",
"node-red-contrib-powershell": "0.0.3",
"node-red-contrib-reusable-flows": "^1.0.0",
"node-red-contrib-salesforce-fix": "catalog:api",
"node-red-contrib-semaphore": "~1.0.1",
"node-red-node-daemon": "~0.4.0",
"oracledb": "catalog:db",
"xml2js": "^0.4.23"
}
}
@moonrockfamily did you have lockfiles in packages folders (outside of the isolate directories)? I think in a monorepo setup there should be only one ever, in the root.
Maybe there's some sort of version mismatch going on. I can imagine that if the lockfile was created using an older version, and isolate-package detects a newer pnpm version, it might invoke the newer prune-lockfile version without respecting the version recorded in the lockfile itself.
But I'm just speculating here :)
@moonrockfamily @insanity54 If you still have a way to reproduce your error, please give 1.20.0-0 a try and let me know if it solved things. You can install it with @next
This version now respects/preserves the actual lockfile version when pruning, instead of going by the declared version in the packageManager field of the manifest.
@moonrockfamily have you been able to test it on your initial problem?
I have been using 12.0.0-0 in my own projects and it seems at least it didn't break anything so I will release it soon. Wish I had some proper tests to cover this 👀
I tried to reproduce with 1.19 but couldn't find the recipe to make a reproducible failure!
I can still reproduce the error on 1.19.0 and 1.20.0-0. I made a reproduction repo
https://github.com/insanity54/isolate-issue-97
Thanks @insanity54 I will get back to it in a week or two to see if I can find the real cause then
So "a week or two" never happened and there's too much other stuff going on 😅
I just wanted to mention that I just ran into the same error message, when I did a find-replace on a monorepo package name. Deleting the lockfile and re-generating it fixed the issuse.
Please give the latest version a try an see if you still have an issue. I didn't fix anything specifically. Closing this for now but feel free to re-open if you run into it still.