meteor
meteor copied to clipboard
meteor
pnpm does not work with Meteor
I was trying out pnpm, but couldn't get it to work out of the box with Meteor.
It's perhaps a compatibility issue in the way pnpm organizes packages in node_modules and how Meteor bundles those packages.
Reproduce
> meteor create pnpm-testapp
> cd testapp
> rm -rf node_modules
> rm package-lock.json
> meteor npm install -g pnpm
> meteor pnpm install
> meteor
...and you get this:
Stacktrace
=> Started proxy.
W20200208-13:22:55.314(2)? (STDERR) /Users/arggh/.meteor/packages/meteor-tool/.1.9.0.aoqmdf.9052m++os.osx.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.osx.x86_64/dev_bundle/server-lib/node_modules/fibers/fibers.js:90
W20200208-13:22:55.357(2)? (STDERR) return fn.apply(this, arguments);
W20200208-13:22:55.357(2)? (STDERR) ^
W20200208-13:22:55.357(2)? (STDERR)
W20200208-13:22:55.357(2)? (STDERR) TypeError: Cannot read property 'indexOf' of undefined
W20200208-13:22:55.358(2)? (STDERR) at module (packages/babel-runtime.js:31:26)
W20200208-13:22:55.358(2)? (STDERR) at fileEvaluate (packages/modules-runtime.js:336:7)
W20200208-13:22:55.358(2)? (STDERR) at Module.require (packages/modules-runtime.js:238:14)
W20200208-13:22:55.358(2)? (STDERR) at require (packages/modules-runtime.js:258:21)
W20200208-13:22:55.358(2)? (STDERR) at packages/babel-runtime.js:53:15
W20200208-13:22:55.358(2)? (STDERR) at packages/babel-runtime.js:58:3
W20200208-13:22:55.359(2)? (STDERR) at /Users/arggh/Development/sandbox/pnpm-test/.meteor/local/build/programs/server/boot.js:398:38
W20200208-13:22:55.359(2)? (STDERR) at Array.forEach (<anonymous>)
W20200208-13:22:55.359(2)? (STDERR) at /Users/arggh/Development/sandbox/pnpm-test/.meteor/local/build/programs/server/boot.js:226:21
W20200208-13:22:55.359(2)? (STDERR) at /Users/arggh/Development/sandbox/pnpm-test/.meteor/local/build/programs/server/boot.js:461:7
=> Exited with code: 1
=> Your application is crashing. Waiting for file change.
I've been upgrading an old app to NPM packages and I've run into several very recent NPM packages that just won't work. Mostly complaining about unknown variable export. I'm on Meteor 1.8.1. I think this was addressed in Meteor 1.8.2+.
The good news is about 90% of the old Atmosphere packages and vanilla JS libraries' NPM packages worked great.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Hi @arggh I think the question here is how many Meteor users are using pnpm? Or what is the adoption of pnpm in the market as a whole?
Hi @arggh I think the question here is how many Meteor users are using pnpm? Or what is the adoption of pnpm in the market as a whole?
I have no answer regarding the adoption among Meteor developers or developers in general, except for the 85,444 total weekly downloads from npm, contrasted with 2 million weekly downloads of npm (~4%).
Even with the relatively low (is it?) adoption, I think Meteor's bundler should work with pnpm - others do. Pnpm isn't doing anything non-standard, as far as I know.
I think pnpm is growing in popularity fast and it has some benefits not yet found in other package managers. I know many very talented developers who prefer it over yarn or npm. I wouldn't want to deter those developers from Meteor by excluding their favorite package manager from the Meteor ecosystem.
Having the option to switch between package managers has also saved me multiple times. Npm has been riddled with bugs lately and the option to switch over to yarn has been a life-saver for me!
Hi @arggh I think the question here is how many Meteor users are using pnpm? Or what is the adoption of pnpm in the market as a whole?
Zero I imagine, as it's not working. Meteor not working with pnpm prevents us from adopting it for our monorepo.
EDIT: To expand on this issue, Meteor's non-standard module resolution is also causing issues in other places (such as in monorepos where dependencies are hoisted to the root of the repository). I imagine that if Meteor would simply adhere to the established standards pnpm's module structure would work fine.
If I could, I would post this issue again. We have a monorepo with multiple apps and packages.
Two of them Meteor apps. They don't play along with the others. Such a pain to setup every time.
Tried it today with Meteor 2.2 and it is still an issue.
Stacktrace
➜ pnpm-testapp meteor
[[[[[ ~/Web/meteor/pnpm-testapp ]]]]]
=> Started proxy.
=> Started HMR server.
Unable to resolve some modules:
"object-assign" in /home/storyteller/Web/meteor/pnpm-testapp/node_modules/react/cjs/react.production.min.js (web.browser)
"scheduler" in /home/storyteller/Web/meteor/pnpm-testapp/node_modules/react-dom/cjs/react-dom.production.min.js (web.browser)
"scheduler/tracing" in /home/storyteller/Web/meteor/pnpm-testapp/node_modules/react-dom/cjs/react-dom.development.js (web.browser)
If you notice problems related to these missing modules, consider running:
meteor npm install --save object-assign scheduler
Unable to resolve some modules:
"object-assign" in /home/storyteller/Web/meteor/pnpm-testapp/node_modules/react/cjs/react.production.min.js (web.browser.legacy)
"scheduler" in /home/storyteller/Web/meteor/pnpm-testapp/node_modules/react-dom/cjs/react-dom.production.min.js (web.browser.legacy)
"scheduler/tracing" in /home/storyteller/Web/meteor/pnpm-testapp/node_modules/react-dom/cjs/react-dom.development.js (web.browser.legacy)
If you notice problems related to these missing modules, consider running:
meteor npm install --save object-assign scheduler
=> Started MongoDB.
I20210423-10:39:32.879(2)? Exception from task TypeError: _objectSpread is not a function
I20210423-10:39:32.923(2)? at Function.Log.<computed> [as error] (packages/logging/logging.js:173:8)
I20210423-10:39:32.923(2)? at packages/webapp/webapp_server.js:636:13
I20210423-10:39:32.923(2)? at runWithEnvironment (packages/meteor.js:1286:24)
I20210423-10:39:32.924(2)? at Object.task (packages/meteor.js:1299:14)
I20210423-10:39:32.924(2)? at Meteor._SynchronousQueue.SQp._run (packages/meteor.js:917:16)
I20210423-10:39:32.924(2)? at packages/meteor.js:894:12
W20210423-10:39:32.924(2)? (STDERR) /home/storyteller/.meteor/packages/meteor-tool/.2.2.0.1n73gp5.cf72++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:313
W20210423-10:39:32.925(2)? (STDERR) throw(ex);
W20210423-10:39:32.925(2)? (STDERR) ^
W20210423-10:39:32.925(2)? (STDERR)
W20210423-10:39:32.925(2)? (STDERR) TypeError: _objectSpread is not a function
W20210423-10:39:32.925(2)? (STDERR) at Function.Log.<computed> [as error] (packages/logging/logging.js:173:8)
W20210423-10:39:32.925(2)? (STDERR) at packages/webapp/webapp_server.js:636:13
W20210423-10:39:32.925(2)? (STDERR) at runWithEnvironment (packages/meteor.js:1286:24)
W20210423-10:39:32.925(2)? (STDERR) at Object.task (packages/meteor.js:1299:14)
W20210423-10:39:32.925(2)? (STDERR) at Meteor._SynchronousQueue.SQp._run (packages/meteor.js:917:16)
W20210423-10:39:32.925(2)? (STDERR) at packages/meteor.js:894:12
=> Exited with code: 1
=> Your application is crashing. Waiting for file change.
I would suggest to ask a different question @filipenevola : What is the reason pnpm should be compatible with any project but not Meteor, as long as it provides the same features and public API as any other alternative package manager? You see, great things are not always as popular as they would deserve. pnpm is an excellent tool and IMHO it is what npm should have been instead of duplicating everything. What they did with pnpm is ingenious and pity it's not an option with Meteor.
Actually pnpm works with Meteor you just have to set the node-linker to hoisted. Which preserves legacy behavior by npm.
.npmrc
node-linker=hoisted
We want to have a simple monorepo with (p)npm workspaces but we can't get npm and pnpm to work.
We have the following structure:
- apps
- myapp
- depends on: package-a
- depends on:
@snowplow/browser-tracker
- myapp
- packages
- package-a
- depends on package b
- package-b
- package-a
pnpm-workspace.yaml
packages:
- 'packages/**'
- 'apps/**'
We can get the local packages working but we have trouble with the snowplow package in our meteor app (or other package which have dependencies).
When we run meteor we get the error:
Unable to resolve some modules: "@snowplow/browser-tracker-core" in ...
This dependency is a dependency of @snowplow/browser-tracker which Meteor seems to have trouble resolving.
We also get this problem with node-linker=hoisted in our .npmrc.
@timvansteenis if you can create a repo that replicates the problem, I'll look into it.
Hi @harryadel , thanks for your fast response and your willingness to help us. We appreciate that!
I created a repo: https://github.com/lessonup/meteor-monorepo-issue
Eager to know if you see a solution. Note that we also tried NPM workspaces with a similar setup, but without success.
@timvansteenis Thanks for putting it together, I like those who put effort when asking for help :). I created an issue. Please let me know what you think.
For the rest who want to pair pnpm with Meteor in a workspace, be sure to set shared-workspace-lockfile to false.
@StorytellerCZ
@harryadel so the simple fix is to update documentation, I guess.
Eh, not really. Documenting the use of node-linker=hoisted gets you working but the real issue is Meteor's inability to work when node_modules are symlinked from a different place which is what pnpm does by default.
isolated - dependencies are symlinked from a virtual store at node_modules/.pnpm.
@harryadel thanks to your help we had a working workspace for a moment, but we encountered a new issue as soon as I added a hook. I described the issue in the readme at https://github.com/lessonup/meteor-monorepo-issue#react-is-being-imported-twice
It seems react is being imported twice because of the hoisted structure. I'm hoping you have any ideas how we can solve that.
@timvansteenis I've been struggling with exactly the same issue recently and I believe I have a solution candidate here:
https://github.com/apendua/meteor/tree/resolve-symlinks-v2-2.7.2
It's a WIP quality, so please do not try to use it for production builds but I'd appreciate any help in testing the new code. I have tried running it against your example repository and it seems to be working fine, at least locally.
In order to activate the new behavior for symbolic links resolution inside node_modules you will need to:
export METEOR_MONOREPO_ROOT=[...]
before you execute meteor run. I know it's a little bit inconvenient and I am planning to change it down the road to be a CLI option.
cc @harryadel
@apendua sorry for my late reply, I wasn't available for a long time.
Thanks for your effort, it would be really great if it would work, we currently have some workarounds in place to make use of a monorepo package structure, but we would love to get rid of the workarounds.
I might have time next week to run your build on our applications and test if they work. I'll get back to you.
@timvansteenis
I might have time next week to run your build on our applications and test if they work. I'll get back to you.
Thanks! I am looking forward to see the results.
@apendua Sorry for my late reply. Was gone for a while.
I can't get my example repo to work when I
- remove the
shared-workspace-lockfilesettings in https://github.com/lessonup/meteor-monorepo-issue/blob/main/.npmrc - run
pnpm iin the root - run
METEOR_MONOREPO_ROOT=/Users/tims/Projects/lessonup/meteor-monorepo-issue ~/Projects/meteor/meteor/meteor run(obviously only works on my machine).
I can confirm i'm on the correct branch resolve-symlinks-v2-2.7.2
I'm getting the following error:
=> Running Meteor from a checkout -- overrides project version (Meteor 2.7.2)
[[[[[ ~/Projects/lessonup/meteor-monorepo-issue/apps/myapp ]]]]]
=> Started proxy.
=> Started HMR server.
Unable to resolve some modules:
"react" in /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/client/main.tsx (web.browser)
"react-dom" in /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/client/main.tsx (web.browser)
"@snowplow/browser-tracker" in /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/imports/ui/App.tsx (web.browser)
If you notice problems related to these missing modules, consider running:
meteor npm install --save react react-dom @snowplow/browser-tracker
Unable to resolve some modules:
"@babel/runtime/helpers/slicedToArray" in /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/imports/ui/App.tsx (web.browser.legacy)
"react" in /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/client/main.tsx (web.browser.legacy)
"react-dom" in /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/client/main.tsx (web.browser.legacy)
"@snowplow/browser-tracker" in /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/imports/ui/App.tsx (web.browser.legacy)
If you notice problems related to these missing modules, consider running:
meteor npm install --save @babel/runtime react react-dom @snowplow/browser-tracker
=> Started MongoDB.
W20221118-14:08:40.393(1)? (STDERR) /Users/tims/Projects/meteor/meteor/dev_bundle/server-lib/node_modules/fibers/fibers.js:92
W20221118-14:08:40.402(1)? (STDERR) return fn.apply(this, arguments);
W20221118-14:08:40.402(1)? (STDERR) ^
W20221118-14:08:40.402(1)? (STDERR)
W20221118-14:08:40.402(1)? (STDERR) Error:
W20221118-14:08:40.402(1)? (STDERR) The @babel/runtime npm package could not be found in your node_modules
W20221118-14:08:40.402(1)? (STDERR) directory. Please run the following command to install it:
W20221118-14:08:40.403(1)? (STDERR)
W20221118-14:08:40.403(1)? (STDERR) meteor npm install --save @babel/runtime
W20221118-14:08:40.403(1)? (STDERR)
W20221118-14:08:40.403(1)? (STDERR) at module (packages/babel-runtime.js:20:9)
W20221118-14:08:40.403(1)? (STDERR) at fileEvaluate (packages/modules-runtime.js:336:7)
W20221118-14:08:40.403(1)? (STDERR) at Module.require (packages/modules-runtime.js:238:14)
W20221118-14:08:40.403(1)? (STDERR) at require (packages/modules-runtime.js:258:21)
W20221118-14:08:40.403(1)? (STDERR) at packages/babel-runtime.js:53:15
W20221118-14:08:40.403(1)? (STDERR) at packages/babel-runtime.js:58:3
W20221118-14:08:40.404(1)? (STDERR) at /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/.meteor/local/build/programs/server/boot.js:401:38
W20221118-14:08:40.404(1)? (STDERR) at Array.forEach (<anonymous>)
W20221118-14:08:40.404(1)? (STDERR) at /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/.meteor/local/build/programs/server/boot.js:226:21
W20221118-14:08:40.404(1)? (STDERR) at /Users/tims/Projects/lessonup/meteor-monorepo-issue/apps/myapp/.meteor/local/build/programs/server/boot.js:464:7
Hey guys, I've used @harryadel suggestions and created a repo with Nx + Pnpm workspaces + Meteor you can check the following repo here: https://github.com/henriquealbert/meteor-pnpm
@timvansteenis It took me a while but I've finally put together a solution that could hopefully work for you. Please have a look at my fork of your example repo:
https://github.com/apendua/meteor-monorepo-issue