unum-cloud
Bug: [email protected] nodejs installation fails
Describe the bug
NPM install fails on arm64 linux. I presume it couldn’t find a new prebuilt binary (exciting new feature from @sroussey), and fell back to compiling from scratch, which it can no longer do because the source and gyp files have been removed from the npm package.
Run npm ci
npm WARN deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
npm WARN deprecated @aws-sdk/[email protected]: This package has moved to @smithy/middleware-retry
npm WARN deprecated @aws-sdk/[email protected]: This package has moved to @smithy/config-resolver
npm WARN deprecated [email protected]: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated @aws-sdk/[email protected]: This package has moved to @smithy/smithy-client
npm WARN deprecated @astrojs/[email protected]: This package is not used by Astro any more and is no longer maintained. In Astro 3.0 polyfills are part of a core module.
npm ERR! code 1
npm ERR! path /home/runner/actions-runner/_work/project/project/node_modules/usearch
npm ERR! command failed
npm ERR! command sh -c node-gyp-build
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | arm64
npm ERR! gyp info find Python using Python version 3.10.12 found at "/usr/bin/python3"
npm ERR! gyp http GET https://nodejs.org/download/release/v20.11.1/node-v20.11.1-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v20.11.1/node-v20.11.1-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v20.11.1/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v20.11.1/SHASUMS256.txt
npm ERR! gyp info spawn /usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/home/runner/actions-runner/_work/project/project/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/runner/actions-runner/_work/project/project/node_modules/usearch/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/runner/actions-runner/_work/project/project/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/runner/.cache/node-gyp/20.11.1/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/home/runner/.cache/node-gyp/20.11.1',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/home/runner/actions-runner/_work/project/project/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/home/runner/.cache/node-gyp/20.11.1/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/home/runner/actions-runner/_work/project/project/node_modules/usearch',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /home/runner/actions-runner/_work/project/project/node_modules/usearch) while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.<anonymous> (/home/runner/actions-runner/_work/project/project/node_modules/node-gyp/lib/configure.js:297:18)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:518:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Linux 5.15.0-84-generic
npm ERR! gyp ERR! command "/opt/hostedtoolcache/node/20.11.1/arm64/bin/node" "/home/runner/actions-runner/_work/project/project/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/runner/actions-runner/_work/project/project/node_modules/usearch
npm ERR! gyp ERR! node -v v20.11.1
npm ERR! gyp ERR! node-gyp -v v10.1.0
npm ERR! gyp ERR! not ok
npm ERR! A complete log of this run can be found in: /home/runner/.npm/_logs/2024-04-01T21_50_25_304Z-debug-0.log
Error: Process completed with exit code 1.
Steps to reproduce
On an arm64 machine, run
docker run -it --entrypoint /bin/bash --rm public.ecr.aws/lambda/nodejs:20-arm64
Inside the container, run
dnf install -y python3 make gcc-c++
npm init -y
npm i usearch
View the error:
npm ERR! code 1
npm ERR! path /var/task/node_modules/usearch
npm ERR! command failed
npm ERR! command sh -c node-gyp-build
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | arm64
npm ERR! gyp info find Python using Python version 3.9.16 found at "/usr/bin/python3"
npm ERR! gyp http GET https://nodejs.org/download/release/v20.9.0/node-v20.9.0-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v20.9.0/node-v20.9.0-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v20.9.0/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v20.9.0/SHASUMS256.txt
npm ERR! gyp info spawn /usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/var/lang/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/var/task/node_modules/usearch/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/var/lang/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/root/.cache/node-gyp/20.9.0/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/20.9.0',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/var/lang/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/20.9.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/var/task/node_modules/usearch',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /var/task/node_modules/usearch) while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.onCpExit (/var/lang/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:325:16)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:514:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Linux 6.6.12-linuxkit
npm ERR! gyp ERR! command "/var/lang/bin/node" "/var/lang/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /var/task/node_modules/usearch
npm ERR! gyp ERR! node -v v20.9.0
npm ERR! gyp ERR! node-gyp -v v9.4.0
npm ERR! gyp ERR! not ok
npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-04-01T23_50_47_987Z-debug-0.log
Expected behavior
NPM install to work with the linux-arm64 prebuilds
Regardless, it’s important for falling back to compiling from scratch with node-gyp to continue to work.
USearch version
2.10.x
Operating System
Amazon Linux 2023
Hardware architecture
Arm
Which interface are you using?
Other bindings
Contact Details
Ping me in Discord
Is there an existing issue for this?
- [X] I have searched the existing issues
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
That's good to know! Another related issue I am solving is the lack of Doxygen support for TypeScript. Maybe we should provide both binding.gyp and prebuilt, as well as TypeScript + JavaScript?
It is odd since there should be a pre-built binary for linux-arm64
https://www.npmjs.com/package/usearch?activeTab=code
:tada: This issue has been resolved in version 2.10.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:
@jlarmstrongiv, sadly, the build fails. @sroussey, would you be able to double-check the cross-compilation commands? I am also looking to extend cross-compilation in Rust as well (for #378).
I will try it as soon as 2.10.4 is released on npm 😄
Try npm install @sroussey/usearch
as a test.
bash-5.2# npm install @sroussey/usearch
added 5 packages, and audited 6 packages in 6s
found 0 vulnerabilities
The install passed! Haven’t tried running sample code yet though
@jlarmstrongiv, sadly, the build fails. @sroussey, would you be able to double-check the cross-compilation commands? I am also looking to extend cross-compilation in Rust as well (for #378).
Might remove the cross compile for now while I do more investigation.
How is it going, @sroussey? How should I change the CI for builds to pass?
I’m home a bit today, so I’ll comment out that section until I have a working version.
See https://github.com/unum-cloud/usearch/commit/c6fb87574f0410cd50b4939b856b1cdbacec42cc
It has what i think is the fix, but I am not setup to test it.
So farther down I have it skip the cross compiling.
I have a docker setup on x86_64 but I can't get it to compile normally, so I can't test cross compiling.
@ashvardanian Have any idea how i set things up wrong?
root@6a1e9ab171d0:/usr/usearch# uname -a
Linux 6a1e9ab171d0 6.4.16-linuxkit #1 SMP PREEMPT Thu Nov 16 10:49:20 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
apt install -y cmake build-essential libjemalloc-dev libomp-dev gcc-12 g++-12
apt install -y gcc-aarch64-linux-gnu binutils-aarch64-linux-gnu g++-aarch64-linux-gnu
root@6a1e9ab171d0:/usr/usearch# git submodule
0a92994d729ff76a58f692d3028ca1b64b145d91 fp16 (heads/master)
127ead1da7c39957b30a50dd85e74814edb022d6 simsimd (v4.2.2)
c2848c1ba777c541af2903501f3d6344d711638b stringzilla (v3.7.0)
The _ph suffixes are for half-precision. GCC 12 must support it. You download it, but seems like you are not using it, @sroussey 🤷
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 60 --slave /usr/bin/g++ g++ /usr/bin/g++-12 fixed it, thanks.
Oh, it did build it:
root@6a1e9ab171d0:/usr/usearch# file build/Release/obj.target/usearch.node
build/Release/obj.target/usearch.node: ELF 64-bit LSB shared object, ARM aarch64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=c0650f24bd82a89cca3a4a773455913f2477a789, not stripped
It just didn't get moved over. I wonder if a strip issue.
Indeed, something went wrong with strip. Manually works though. I can work around this...
Hmm, I get errors that:
Error: src/usearch.ts(11,5): error TS2322: Type '"cos"' is not assignable to type 'MetricKind'.
Error: src/usearch.ts(13,5): error TS2322: Type '"f32"' is not assignable to type 'ScalarKind'.
Did the types change?
It seems my bundling for usearch does not work with the new package structure. I am using https://www.npmjs.com/package/@vercel/ncc, I wonder why that would break and not detect the usearch.node file anymore
Uncaught Exception {
"errorType": "Error",
"errorMessage": "Could not find native build for usearch",
"stack": [
"Error: Could not find native build for usearch",
" at getBuildDir (/opt/nodejs/node_modules/usearch/index.js:1080:15)",
" at getBuildDir (/opt/nodejs/node_modules/usearch/index.js:1081:12)",
" at getBuildDir (/opt/nodejs/node_modules/usearch/index.js:1081:12)",
" at getBuildDir (/opt/nodejs/node_modules/usearch/index.js:1081:12)",
" at getBuildDir (/opt/nodejs/node_modules/usearch/index.js:1081:12)",
" at 128 (/opt/nodejs/node_modules/usearch/index.js:605:48)",
" at __nccwpck_require__ (/opt/nodejs/node_modules/usearch/index.js:1117:43)",
" at /opt/nodejs/node_modules/usearch/index.js:1137:37",
" at Object.<anonymous> (/opt/nodejs/node_modules/usearch/index.js:1140:12)",
" at Module._compile (node:internal/modules/cjs/loader:1376:14)"
]
}
These tools are not smart enough to deal with .node or .wasm files (generally). They will require extra steps on your side.