sharp icon indicating copy to clipboard operation
sharp copied to clipboard

Sharp Loader Prohibits Platform-Specific Compiled Sharp from Loading

Open chlarsen opened this issue 3 months ago • 13 comments

It seems that your loader procactively prohibits the use of Sharp on FreeBSD, which, quite frankly does not make any sense whatsoever. Steps to reproduce:

pkg install vips
vips --version
vips-8.17.2

Next, in a dedicated folder and as non-privileged user:

export PKG_CONFIG_PATH="/usr/local/lib/pkgconfig"
export CXX=clang++
export CXXFLAGS="-std=c++17 -stdlib=libc++"
export CPPFLAGS="-I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include"
export LDFLAGS="-L/usr/local/lib -lvips -lvips-cpp"
export SHARP_BUILD_FROM_SOURCE=true

npm install node-addon-api node-gyp libvips tslib
npm install [email protected]
cd ./node_modules/sharp/src
npx node-gyp rebuild
cd /usr/home/sync-in/node_modules/sharp
mv src/build build

Note: Version-pinning of Sharp is needed to maintain compatibility with the latest avaiable vips port on FreeBSD. Testing:

node -e "
const binary = require('./node_modules/sharp/build/Release/sharp-freebsd-x64.node');
console.log('Libvips version from binary:', binary.libvipsVersion());
console.log('Sharp should work - the loader is the problem!');

And:

# Check what sharp's loader is complaining about
node -e "
const help = [
  'Could not load the \"sharp\" module using the freebsd-x64 runtime',
  'Possible solutions:',
  '- Manually install libvips >= 8.17.2', 
  '- Add experimental WebAssembly-based dependencies:',
  '    npm install --cpu=wasm32 sharp',
  '    npm install @img/sharp-wasm32',
  '- Consult the installation documentation:',
  '    See https://sharp.pixelplumbing.com/install'
];
console.log('Loader error message is hardcoded!');
console.log('The loader is not even checking if the binary works!');
"

Well, the loader just peeps at the OS, and bails out, even without testing my shiny binary. This is not quite what the doctor ordered, is it? Please allow those on FreeBSD to use your software! The fix should be easy; I have for now cut out the IMHO over-designed testing and issued:

cd
cp ~/node_modules/sharp/lib/sharp.js ~/node_modules/sharp/lib/sharp.js.orig
cat > ~/node_modules/sharp/lib/sharp.js << EOF
// Copyright 2013 Lovell Fuller and others.
// SPDX-License-Identifier: Apache-2.0

'use strict';

// FreeBSD patch: directly use our manually built binary
const { runtimePlatformArch } = require('./libvips');
const runtimePlatform = runtimePlatformArch();

module.exports = require(`../src/build/Release/sharp-${runtimePlatform}.node`);
EOF
node -e "require('sharp'); console.log('✅ Sharp loaded with nuclear patch!')"

Tadaaaa! Thoughts regarding my urgent sugggestion to liberalise your user pool :-) ?

chlarsen avatar Nov 12 '25 08:11 chlarsen

The FreeBSD CI environment is currently failing due to a lack of credits, but the most recent passing build worked.

https://cirrus-ci.com/task/6141689489784832

What is the output of running npm install --verbose --foreground-scripts [email protected]?

lovell avatar Nov 12 '25 12:11 lovell

With:

$ npm install --verbose --foreground-scripts [email protected]

I get:

npm verbose cli /usr/local/bin/node /usr/local/bin/npm
npm info using [email protected]
npm info using [email protected]
npm warn Unknown global config "python". This will stop working in the next major version of npm.
npm verbose title npm install [email protected]
npm verbose argv "install" "--loglevel" "verbose" "--foreground-scripts" "[email protected]"
npm verbose logfile logs-max:10 dir:/home/sync-in-test/.npm/_logs/2025-11-13T11_06_50_370Z-
npm verbose logfile /home/sync-in-test/.npm/_logs/2025-11-13T11_06_50_370Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/npm 203ms
npm http fetch GET 200 https://registry.npmjs.org/sharp 352ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver 52ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fcolour 59ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc 71ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm 88ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-ia32 52ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-x64 58ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm64 51ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-ppc64 60ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-arm64 51ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-s390x 74ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-arm64 64ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-x64 47ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-arm64 46ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-x64 232ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-x64 46ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-x64 58ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm64 44ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-ppc64 46ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-arm64 45ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-x64 54ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-s390x 66ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm 138ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-arm64 49ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-x64 334ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-wasm32 652ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@emnapi%2fruntime 68ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tslib 38ms (cache miss)
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-win32-x64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-win32-ia32
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-win32-arm64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-wasm32
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@emnapi/runtime
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/tslib
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-linuxmusl-x64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-linuxmusl-arm64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-linux-x64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-linux-s390x
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-linux-ppc64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-linux-arm64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-linux-arm
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-linuxmusl-x64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-linuxmusl-arm64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-linux-x64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-linux-s390x
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-linux-ppc64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-linux-arm64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-linux-arm
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-darwin-x64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-libvips-darwin-arm64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-darwin-x64
npm verbose reify failed optional dependency /usr/home/sync-in-test/node_modules/@img/sharp-darwin-arm64
npm http cache tslib@https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz 1ms (cache hit)
npm http cache semver@https://registry.npmjs.org/semver/-/semver-7.7.3.tgz 0ms (cache hit)
npm http cache @img/colour@https://registry.npmjs.org/@img/colour/-/colour-1.0.0.tgz 0ms (cache hit)
npm http cache sharp@https://registry.npmjs.org/sharp/-/sharp-0.34.4.tgz 0ms (cache hit)
npm http cache detect-libc@https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz 0ms (cache hit)
npm http cache @emnapi/runtime@https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.7.0.tgz 0ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@img/colour/-/colour-1.0.0.tgz 69ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz 70ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz 73ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.7.0.tgz 91ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver/-/semver-7.7.3.tgz 103ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/sharp/-/sharp-0.34.4.tgz 106ms (cache miss)
npm info run [email protected] install node_modules/sharp node install/check.js

> [email protected] install
> node install/check.js

npm info run [email protected] install { code: 0, signal: null }

added 28 packages in 3s

23 packages are looking for funding
  run `npm fund` for details
npm verbose cwd /usr/home/sync-in-test
npm verbose os FreeBSD 14.3-RELEASE-p5
npm verbose node v22.19.0
npm verbose npm  v11.6.1
npm notice
npm notice New patch version of npm available! 11.6.1 -> 11.6.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.6.2
npm notice To update run: npm install -g [email protected]
npm notice
npm verbose exit 0
npm info ok

Inside the ./node_modules/sharp folder we have:

$ ls -l ./node_modules/sharp
total 31
-rw-r--r--  1 sync-in-test sync-in-test 10273 Nov 13 11:06 LICENSE
-rw-r--r--  1 sync-in-test sync-in-test  3160 Nov 13 11:06 README.md
drwxr-xr-x  2 sync-in-test sync-in-test     3 Nov 13 11:06 install
drwxr-xr-x  2 sync-in-test sync-in-test    16 Nov 13 11:06 lib
-rw-r--r--  1 sync-in-test sync-in-test  7694 Nov 13 11:06 package.json
drwxr-xr-x  2 sync-in-test sync-in-test    16 Nov 13 11:06 src

So, as before, no building done.

chlarsen avatar Nov 13 '25 11:11 chlarsen

Thanks, this tells me it was unable to find a globally-installed vips package of the required minimum (for sharp v0.34.4 this will be libvips v8.17.2).

What is the output of pkg-config --modversion vips-cpp?

lovell avatar Nov 13 '25 11:11 lovell

Thanks, this tells me it was unable to find a globally-installed vips package of the required minimum (for sharp v0.34.4 this will be libvips v8.17.2).

What is the output of pkg-config --modversion vips-cpp?

This is not the reason. vips is installed with the right version. the issue is that you loader refused freebsd and that a compilation from source is needed:

### Linux no have :-)
pkg-config --modversion vips-cpp
pkg-config: Command not found.
(pts/1)[root@fileserver:~]# vips --version
vips-8.17.2

chlarsen avatar Nov 13 '25 12:11 chlarsen

I'm not sure I understand what you mean by "refused freebsd".

Whilst prebuilt binaries are made available for many platforms, they are not currently provided for FreeBSD. You must therefore provide a globally-installed libvips so sharp can be compiled from source against it.

https://sharp.pixelplumbing.com/install/#freebsd

https://sharp.pixelplumbing.com/install/#building-from-source

sharp uses pkg-config to determine the availability and location of the globally-installed libvips. Perhaps you need to install both the devel/pkgconf and graphics/vips packages? See the FreeBSD CI configuration.

lovell avatar Nov 13 '25 12:11 lovell

AFAIK, the pkg-config executable has been removed from FreeBSD donkey years ago, even though it is still in man. Correct me, if I am wrong.

chlarsen avatar Nov 13 '25 12:11 chlarsen

I'm referring to the bin/pkg-config executable installed by the pkgconf package (I think it might be a symlink to the bin/pkgconf executable, or vice versa).

lovell avatar Nov 13 '25 20:11 lovell

You are correct: The rather non-intuitive environment settings for compilation from source can be avoided by installing devel/pkgconf beforehand. In summary, the FreeBSD installation process is as follows:

As root user:

pkg install devel/pkgconf graphics/vips
vips --version

Make a note of the vips version installed and install the corresponding sharp version. In our setting we have:

vips version: 8.17.2 sharp version: 0.34.4

Proceed as non-privileged use that is meant to run sync-in:

SHARP_VERSION=[0.34.4]
npm install node-addon-api node-gyp libvips tslib
npm install sharp@${SHARP_VERSION}
cd ./node_modules/sharp/src
npx node-gyp rebuild
cd

Note: It seems to be essential to install libvips and tslib explicitly prior to the rest. Test, whether sharp is working:

node -e "require('sharp'); console.log('✅ Sharp loaded!')"

Thank you!

chlarsen avatar Nov 14 '25 03:11 chlarsen

pkg install devel/pkgconf graphics/vips

Yes, this matches the documentation - https://sharp.pixelplumbing.com/install/#freebsd

npm install node-addon-api node-gyp libvips tslib

Yes to installing node-addon-api and node-gyp - this matches the documentation https://sharp.pixelplumbing.com/install/#building-from-source

However also no, you do not need to install the tslib and libvips packages from npm.

npx node-gyp rebuild

No, you do not need to do this, as npm install sharp@${SHARP_VERSION} should detect the globally-installed libvips and run it for you.

lovell avatar Nov 14 '25 08:11 lovell

I tried without tslib and libvips, and it fails.

On 14 November 2025 08:06:25 UTC, Lovell Fuller @.***> wrote:

lovell left a comment (lovell/sharp#4476)

pkg install devel/pkgconf graphics/vips

Yes, this matches the documentation - https://sharp.pixelplumbing.com/install/#freebsd

npm install node-addon-api node-gyp libvips tslib

No, you do not need to install the tslib and libvips packages from npm.

npx node-gyp rebuild

No, you do not need to do this, as npm install sharp@${SHARP_VERSION} should detect the globally-installed libvips and run it for you.

-- Reply to this email directly or view it on GitHub: https://github.com/lovell/sharp/issues/4476#issuecomment-3531465928 You are receiving this because you authored the thread.

Message ID: @.***>

chlarsen avatar Nov 14 '25 08:11 chlarsen

What is the output of running the following in an empty directory?

pkg-config --modversion vips-cpp
npm install node-addon-api node-gyp
npm install --verbose --foreground-scripts [email protected]

lovell avatar Nov 14 '25 09:11 lovell

I get:

$ pkg-config --modversion vips-cpp
8.17.2
npm install node-addon-api node-gyp
npm warn Unknown global config "python". This will stop working in the next major version of npm.

added 98 packages in 4s

16 packages are looking for funding
  run `npm fund` for details
npm notice
npm notice New patch version of npm available! 11.6.1 -> 11.6.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.6.2
npm notice To update run: npm install -g [email protected]
npm notice

For the output of

npm install --verbose --foreground-scripts [email protected]

please see attached install log. After completion I get:

$ ls -l ./node_modules/sharp/
total 32
-rw-r--r--  1 sync-in-test sync-in-test 10273 Nov 15 09:00 LICENSE
-rw-r--r--  1 sync-in-test sync-in-test  3160 Nov 15 09:00 README.md
drwxr-xr-x  2 sync-in-test sync-in-test     3 Nov 15 09:00 install
drwxr-xr-x  2 sync-in-test sync-in-test    16 Nov 15 09:00 lib
drwxr-xr-x  2 sync-in-test sync-in-test     4 Nov 15 09:00 node-addon-api
-rw-r--r--  1 sync-in-test sync-in-test  7694 Nov 15 09:00 package.json
drwxr-xr-x  3 sync-in-test sync-in-test    17 Nov 15 09:00 src

No error message during installation, but no build folder, and the application (sync-in server) complains bitterly about missing tslib and sharp. The only working avenue seems ot be my suggested installation sequence.

install.log

chlarsen avatar Nov 15 '25 09:11 chlarsen

Thanks, and what is the output of the following (after running the above):

ls -al node_modules/sharp/src/build/*
node -p "require('./node_modules/sharp/src/build/Release/sharp-freebsd-x64.node')"

As an aside, the appearance of the following line in the provided output concerns me, as that should not be there and it's unclear where it came from:

drwxr-xr-x  2 sync-in-test sync-in-test     4 Nov 15 09:00 node-addon-api

lovell avatar Nov 15 '25 10:11 lovell