Serverless fails to bundle modules when using pnpm

[noelforte]

Describe the bug I use pnpm to manage my node_modules and have encountered a bug with Eleventy Serverless that occurs when the project is using pnpm, most likely due to the nature of the structure of node_modules being a virtual store that hard links to a global store.

When a serverless bundle is generated from a project that is using pnpm, the resulting bundle fails to execute properly with the error: [11ty] > Cannot find module '.pnpm'. Further inspection of the error shows that within the bundle itself, in eleventy-app-config-modules.js the following statement is made:

require('.pnpm');

which is different than what's placed there when the project is using npm:

require('@11ty/eleventy');

Raw Output from netlify dev

❯ netlify dev
◈ Netlify Dev ◈
◈ Ignored general context env var: LANG (defined in process)
◈ Loaded function dyn.
◈ Functions server is listening on 34855
◈ Starting Netlify Dev with Eleventy
[11ty] Writing output/static/index.html from ./input/static.njk
[11ty] Serverless: 3 files bundled to ./netlify/functions/dyn.
[11ty] Wrote 1 file in 0.11 seconds (v1.0.0-canary.41)
[11ty] Watching…
◈ Reloading function dyn...
◈ Reloaded function dyn
[Browsersync] Access URLs:
 -----------------------------------
    Local: http://localhost:8080
 External: http://172.30.86.161:8080
 -----------------------------------
[Browsersync] Serving files from: output
◈ Reloading function dyn...
◈ Reloaded function dyn
[11ty] Unhandled rejection in promise: (more in DEBUG output)
[11ty] > Cannot find module '.pnpm'
Require stack:
- /home/noel/projects/eleventy-serverless/netlify/functions/dyn/eleventy-app-config-modules.js
- /home/noel/projects/eleventy-serverless/netlify/functions/dyn/eleventy-bundler-modules.js
- /home/noel/projects/eleventy-serverless/netlify/functions/dyn/index.js
- /home/noel/projects/eleventy-serverless/node_modules/.pnpm/@[email protected]/node_modules/@11ty/eleventy/src/Plugins/ServerlessBundlerPlugin.js
- /home/noel/projects/eleventy-serverless/node_modules/.pnpm/@[email protected]/node_modules/@11ty/eleventy/src/Eleventy.js
- /home/noel/projects/eleventy-serverless/node_modules/.pnpm/@[email protected]/node_modules/@11ty/eleventy/cmd.js

`Error` was thrown:
[11ty]     Error: Cannot find module '.pnpm'
    Require stack:
    - /home/noel/projects/eleventy-serverless/netlify/functions/dyn/eleventy-app-config-modules.js
    - /home/noel/projects/eleventy-serverless/netlify/functions/dyn/eleventy-bundler-modules.js
    - /home/noel/projects/eleventy-serverless/netlify/functions/dyn/index.js
    - /home/noel/projects/eleventy-serverless/node_modules/.pnpm/@[email protected]/node_modules/@11ty/eleventy/src/Plugins/ServerlessBundlerPlugin.js
    - /home/noel/projects/eleventy-serverless/node_modules/.pnpm/@[email protected]/node_modules/@11ty/eleventy/src/Eleventy.js
    - /home/noel/projects/eleventy-serverless/node_modules/.pnpm/@[email protected]/node_modules/@11ty/eleventy/cmd.js
        at Function.Module._resolveFilename (internal/modules/cjs/loader.js:885:15)
        at Function.Module._load (internal/modules/cjs/loader.js:730:27)
        at Module.require (internal/modules/cjs/loader.js:957:19)
        at require (internal/modules/cjs/helpers.js:88:18)
        at Object.<anonymous> (/home/noel/projects/eleventy-serverless/netlify/functions/dyn/eleventy-app-config-modules.js:1:1)
        at Module._compile (internal/modules/cjs/loader.js:1068:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:1097:10)
        at Module.load (internal/modules/cjs/loader.js:933:32)
        at Function.Module._load (internal/modules/cjs/loader.js:774:14)
        at Module.require (internal/modules/cjs/loader.js:957:19)

---

To Reproduce Steps to reproduce the behavior:

1. Install pnpm: $ npm i -g pnpm
2. Move to your project: $ cd <project>
3. Follow steps at: https://www.11ty.dev/docs/plugins/serverless/#usage
4. Bundle serverless function by running eleventy

Expected behavior When running with npm/npx the resulting bundle can be previewed (ie with $ netlify dev) or deployed without issues or errors by including correct dependencies (in this case looks to be missing require('@11ty/eleventy')

Environment:

• OS and Version: Ubuntu 20.04.2 LTS (Focal Fossa) using kernel 5.4.72-microsoft-standard-WSL2
• Eleventy Version: v1.0.0-canary.41

Additional context My next step for trying to tackle this bug is to look into the workarounds documented here: https://pnpm.io/faq#pnpm-does-not-work-with-your-project-here

I'm no Node expert by any means, but this appears to be a direct result of modules not resolving correctly due to a different-than-expected node_modules structure.

[patrickhaug]

Are there any leads for this, I would be happy to support :)

[noelforte]

@patrickhaug Curious to know as well if someone wants to take a lead for using pnpm with the Serverless plugin. I haven't had time or an immediate need to jump on this one but if I start working toward a solution I'll definitely post updates or open a pull 👍

[patrickhaug]

@noelforte Same same. But I guess I can not look into it the next days ..

[noelforte]

Have an update on this. I read through https://pnpm.io/faq#pnpm-does-not-work-with-your-project-here in more detail and after looking though the workarounds, I'm less convinced that this issue has to do with dependency hoisting, as using --shamefully-hoist when running pnpm install does not yield different results. Therefore, I have to assume it's a module resolution issue.

My best guess is that when bundling, the EleventyServerlessBundlerPlugin does not follow hard links, meaning that links to pnpms store aren't fully resolved. It's hard to know for sure without digging in too deeply into the Eleventy code which is again something I haven't had the time to do given that I'm not a developer-by-day.

To reiterate what I think the root of the issue is: When installing/bundling with NPM, the function file for eleventy-app-config-modules.js contains the following:

require("@11ty/eleventy");
require("@11ty/eleventy-img");
require("@swc/core");
require("cssnano");
require("htmlnano");
require("luxon");
require("markdown-it");
require("postcss");
require("postcss-preset-env");
require("posthtml");
require("sass");
require("svgo");
require("yaml");

but when using pnpm the only line in this file is:

require('.pnpm');

Since requiring .pnpm doesn't actually require any modules, the build fails when using pnpm as a package manager, most likely because pnpm doesn't use a flat node-modules structure AND also symlinks dependencies to its virtual store so they can be reused.

It would be really helpful to get some traction on this issue in the coming months for those of us that use pnpm daily. I'd love to submit a pull request, but unfortunately my knowledge of Eleventy's internals and Node only go so deep. Happy to provide further details and/or keep this issue updated as I dig further into it!

[noelforte]

I was able to trace the issue back to @11ty/dependency-tree, which doesn't appear to resolve packages correctly when installed with pnpm. See the open issue at https://github.com/11ty/eleventy-dependency-tree/issues/2 for continuing development on that. Hopefully once that's fixed that'll resolve this issue.

[patrickhaug]

Nice catch!

[TigersWay]

Not sure how many we are using/trying to use pnpm..... So adding 1 😃

[zachleat]

Stale per project slipstream changes with serverless in #3074. Please regroup on #2876 for official pnpm support!