Error with module resolution loading `sharp` native module as dependency via `npm:` specifier

[k-right]

Trying to get payload running on Deno unstable. Module resolution seems to get confused on the sharp module (specifically loading its native component).

deno run --unstable --allow-read --allow-env index.ts

error: Uncaught Error:
Something went wrong installing the "sharp" module

Cannot find module '../build/Release/sharp-darwin-x64.node'
Require stack:
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/index.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/uploads/generateFileData.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/create.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/create.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/index.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/index.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/index.js

Possible solutions:
- Install with verbose logging and look for errors: "npm install --ignore-scripts=false --foreground-scripts --verbose sharp"
- Install for the current darwin-x64 runtime: "npm install --platform=darwin --arch=x64 sharp"
- Consult the installation documentation: https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:34:9)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:37:4)
    at Module._compile (deno:ext/node/02_require.js:717:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:750:12)
    at Module.load (deno:ext/node/02_require.js:634:34)
    at Function.Module._load (deno:ext/node/02_require.js:491:14)
    at Module.require (deno:ext/node/02_require.js:656:21)
    at require (deno:ext/node/02_require.js:790:18)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:8:1)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:425:4)

Minimal example to reproduce:

import express from 'npm:[email protected]'
import payload from 'npm:[email protected]'

const app = express();

payload.init({
  secret: process.env.PAYLOAD_SECRET,
  mongoURL: process.env.MONGO_URL,
  express: app,
});

app.listen(process.env.PAYLOAD_PORT, process.env.PAYLOAD_ADDR, async () => {
  console.log(`listening on http://${process.env.PAYLOAD_ADDR}:${process.env.PAYLOAD_PORT}/`);
});

[k-right]

cd ~/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2
npm i

deno run --unstable --allow-read --allow-env --allow-ffi index.ts

dyld: lazy symbol binding failed: Symbol not found: _pthread_jit_write_protect_supported_np
  Referenced from: /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/build/Release/../.././vendor/8.13.3/darwin-x64/lib/libvips-cpp.42.dylib
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _pthread_jit_write_protect_supported_np
  Referenced from: /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/build/Release/../.././vendor/8.13.3/darwin-x64/lib/libvips-cpp.42.dylib
  Expected in: /usr/lib/libSystem.B.dylib

[1]    48222 abort      deno run --unstable --allow-read --allow-env --allow-ffi index.ts

[littledivy]

@k-right It seems like a recent bug in Sharp: https://github.com/lovell/sharp/issues/3438

[k-right]

@littledivy Yep, looks like it.

Tried same strategies with Docker denoland/deno@latest, Node.js 16.18.1, NPM 8.19.2.

---

Without npm i

deno run --unstable --allow-read --allow-env --allow-ffi main.ts

✅ Granted run access to "/bin/sh".
error: Uncaught Error:
Something went wrong installing the "sharp" module

Cannot find module '../build/Release/sharp-linux-x64.node'
Require stack:
- /deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js
- /deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js
- /deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/index.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/uploads/generateFileData.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/create.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/create.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/index.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/index.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/index.js

Possible solutions:
- Install with verbose logging and look for errors: "npm install --ignore-scripts=false --foreground-scripts --verbose sharp"
- Install for the current linux-x64 runtime: "npm install --platform=linux --arch=x64 sharp"
- Consult the installation documentation: https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:34:9)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:37:4)
    at Module._compile (deno:ext/node/02_require.js:726:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:759:12)
    at Module.load (deno:ext/node/02_require.js:643:34)
    at Function.Module._load (deno:ext/node/02_require.js:500:14)
    at Module.require (deno:ext/node/02_require.js:665:21)
    at require (deno:ext/node/02_require.js:799:18)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:8:1)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:425:4)

---

With npm i

deno run --unstable --allow-read --allow-env --allow-ffi --allow-run --allow-sys main.ts

napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
error: Uncaught TypeError: model.prototype.$__setSchema is not a function
    at Function.compile (file:///deno-dir/npm/registry.npmjs.org/mongoose/6.5.0/lib/model.js:4937:19)
    at Mongoose._model (file:///deno-dir/npm/registry.npmjs.org/mongoose/6.5.0/lib/index.js:601:27)
    at Mongoose.model (file:///deno-dir/npm/registry.npmjs.org/mongoose/6.5.0/lib/index.js:560:27)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/preferences/model.js:27:34)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/preferences/model.js:39:4)
    at Module._compile (deno:ext/node/02_require.js:726:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:759:12)
    at Module.load (deno:ext/node/02_require.js:643:34)
    at Function.Module._load (deno:ext/node/02_require.js:500:14)
    at Module.require (deno:ext/node/02_require.js:665:21)

---

I'm thinking the former error may be caused by the lack of support for napi_add_finalizer as indicated by the latter. Let me know if there's still something to be investigated here, otherwise it's something to keep an eye on for testing if support for napi_add_finalizer is on the roadmap.

[bartlomieju]

Seems this is the same problem as https://github.com/denoland/deno/issues/17085

[bartlomieju]

@k-right it seems this example involves several other packages. Could you try to boil it down into two separate issues - one for sharp support and one for the error coming from mongoose?

[hahnbeelee]

I've seen the sharp error happen on old versions of Mac OS's. I know nothing about deno but just wanted to put that out there in case it helps you guys narrow down what the problem is.

https://developer.apple.com/forums/thread/707916

[bartlomieju]

napi_add_finalizer is now available and working. @k-right any chance you could try this again and let us know if the problem persists?

[zbynekwinkler]

I tried while checking

• #19854

When all binary files are in place, sharp can be imported (meaning I can run deno repl and paste import sharp from 'npm:sharp'; and not get an error).

[bartlomieju]

@zwn by "when all binary files are in place" do you mean that you manually run postinstall script?

[zbynekwinkler]

@bartlomieju I just tried to import sharp and when it said some file is missing I've put it into the place where it was expected.

There were two parts to make it working:

• node_modules/.deno/[email protected]/node_modules/sharp/build/Release/sharp-linux-x64.node
• node_modules/.deno/[email protected]/node_modules/sharp/vendor/*

[birkskyum]

Related to

• #20071

[shriharip]

facing same issue. simple reproducible code is also the same :)

something went wrong installing the "sharp" module

Cannot find module '../build/Release/sharp-darwin-x64.node'
Require stack:
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/sharp/lib/sharp.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/sharp/lib/constructor.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/sharp/lib/index.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/payload/dist/uploads/generateFileData.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/payload/dist/collections/operations/create.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/payload/dist/collections/requestHandlers/create.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/payload/dist/collections/buildEndpoints.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/payload/dist/collections/initHTTP.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/payload/dist/initHTTP.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/[email protected]/node_modules/payload/dist/index.js
- /Users/shrihari/projects

[birkskyum]

This issue specific to sharp can be closed since sharp v0.33.