sst
Sharp error on deployment
Hello I encounter this error when deploying a simple nextjs app
john@JOHN:~/workspace/app$ npx sst deploy --profile prod --stage prod
SST v2.30.3
➜ App: app
Stage: prod
Region: X
Account: X
[WARNING] aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
use {@link Certificate } instead
This API will be removed in the next major release.
Monorepo detected at /home/john/workspace/app
Next.js v13.5.6
OpenNext v2.2.4
┌─────────────────────────────────┐
│ OpenNext — Building Next.js app │
└─────────────────────────────────┘
> @app/[email protected] build
> next build
✓ Creating an optimized production build
✓ Compiled successfully
Skipping validation of types
✓ Linting
✓ Collecting page data
✓ Generating static pages (9/9)
✓ Collecting build traces
✓ Finalizing page optimization
Route (app) Size First Load JS
┌ ○ / 228 B 80.9 kB
├ ○ /_not-found 874 B 81.5 kB
├ ○ /about 204 B 80.8 kB
├ ○ /community-guidelines 205 B 80.8 kB
├ ○ /privacy 204 B 80.8 kB
├ ○ /support 228 B 80.9 kB
└ ○ /terms 204 B 80.8 kB
+ First Load JS shared by all 80.6 kB
├ chunks/1dd3208c-b2615f357c962e21.js 51.1 kB
├ chunks/53-86c33e625b50f980.js 27.5 kB
├ chunks/main-app-ee4882900a7f2a20.js 230 B
└ chunks/webpack-22732b761b854a9f.js 1.86 kB
○ (Static) automatically rendered as static HTML (uses no initial props)
┌──────────────────────────────┐
│ OpenNext — Generating bundle │
└──────────────────────────────┘
Bundling static assets...
Bundling cache assets...
Bundling server function...
Applying plugins:: [opennext-13.5-serverHandler,opennext-13.5-util,opennext-13.5-default] for Next version: ^13.5.6
Open-next plugin opennext-13.5-serverHandler -- Applying override for imports from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-serverHandler -- Applying override for handler from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-default -- Applying override for imports from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for processInternalEvent from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for postProcessResponse from ./default.replacement.js
Open-next plugin opennext-13.5-util -- Applying override for requireHooks from ./13.5/util.js
Open-next plugin opennext-13.5-util -- Applying override for requestHandler from ./util.replacement.js
Bundling revalidation function...
Bundling image optimization function...
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Downloading https://github.com/stechstudio/libvips-lambda/releases/download/v1.0.1/libvips-8.5.5-lambda.tar.gz/v8.14.4/libvips-8.14.4-linux-arm64v8.tar.br
npm ERR! sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: Prebuilt libvips 8.14.4 binaries are not yet available for linux-arm64v8
npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-20T21_43_45_512Z-debug-0.log
node:internal/errors:932
const err = new Error(message);
^
Error: Command failed: npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" [email protected]
at checkExecSyncError (node:child_process:890:11)
at Object.execSync (node:child_process:962:15)
at createImageOptimizationBundle (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:231:8)
at build (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:33:5)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
status: 1,
signal: null,
output: [ null, null, null ],
pid: 10917,
stdout: null,
stderr: null
}
Node.js v21.0.0
Error: There was a problem building the "Site" site.
Need help with this error? Post it in #help on the SST Discord https://sst.dev/discord
Thank you
Also I tried running the failling command it give more info about the error
john@JOHN:~/workspace/app$ npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" [email protected]
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Detected globally-installed libvips v8.14.4
npm ERR! sharp: Building from source via node-gyp
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | x64
npm ERR! gyp info find Python using Python version 3.8.10 found at "/usr/bin/python3"
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Invalid version number: 18
npm ERR! gyp ERR! stack at getNodeDir (/home/john/.nvm/versions/node/v21.0.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:57:25)
npm ERR! gyp ERR! stack at /home/john/.nvm/versions/node/v21.0.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:31:7
npm ERR! gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
npm ERR! gyp ERR! System Linux 5.15.90.1-microsoft-standard-WSL2
npm ERR! gyp ERR! command "/home/john/.nvm/versions/node/v21.0.0/bin/node" "/home/john/.nvm/versions/node/v21.0.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! gyp ERR! node -v v21.0.0
npm ERR! gyp ERR! node-gyp -v v9.4.0
npm ERR! gyp ERR! not ok
npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-20T21_45_32_141Z-debug-0.log
Error seems pretty clear Error: Invalid version number: 18. You are no .nvm/versions/node/v21.0.0
@sladg thank you for you time!
here is the same run failling on Node.js v18.18.2
john@JOHN:~/workspace/app$ yarn sst deploy --profile prod --stage prod
yarn run v1.22.19
$ /home/john/workspace/app/node_modules/.bin/sst deploy --profile prod --stage prod
SST v2.31.0
➜ App: app
Stage: prod
Region: eu-west-3
Account: 004799217655
[WARNING] aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
use {@link Certificate } instead
This API will be removed in the next major release.
Monorepo detected at /home/john/workspace/app
Next.js v13.5.6
OpenNext v2.2.4
┌─────────────────────────────────┐
│ OpenNext — Building Next.js app │
└─────────────────────────────────┘
$ next build
✓ Creating an optimized production build
✓ Compiled successfully
Skipping validation of types
✓ Linting
✓ Collecting page data
✓ Generating static pages (9/9)
✓ Collecting build traces
✓ Finalizing page optimization
Route (app) Size First Load JS
┌ ○ / 227 B 80.7 kB
├ ○ /_not-found 880 B 81.4 kB
├ ○ /about 206 B 80.7 kB
├ ○ /community-guidelines 205 B 80.7 kB
├ ○ /privacy 203 B 80.7 kB
├ ○ /support 227 B 80.7 kB
└ ○ /terms 206 B 80.7 kB
+ First Load JS shared by all 80.5 kB
├ chunks/1dd3208c-b2615f357c962e21.js 50.9 kB
├ chunks/53-86c33e625b50f980.js 27.5 kB
├ chunks/main-app-ee4882900a7f2a20.js 231 B
└ chunks/webpack-22732b761b854a9f.js 1.86 kB
○ (Static) automatically rendered as static HTML (uses no initial props)
┌──────────────────────────────┐
│ OpenNext — Generating bundle │
└──────────────────────────────┘
Bundling static assets...
Bundling cache assets...
Bundling server function...
Applying plugins:: [opennext-13.5-serverHandler,opennext-13.5-util,opennext-13.5-default] for Next version: ^13.5.6
Open-next plugin opennext-13.5-default -- Applying override for imports from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for processInternalEvent from ./default.replacement.js
Open-next plugin opennext-13.5-default -- Applying override for postProcessResponse from ./default.replacement.js
Open-next plugin opennext-13.5-serverHandler -- Applying override for imports from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-serverHandler -- Applying override for handler from ./13.5/serverHandler.js
Open-next plugin opennext-13.5-util -- Applying override for requireHooks from ./13.5/util.js
Open-next plugin opennext-13.5-util -- Applying override for requestHandler from ./util.replacement.js
Bundling revalidation function...
Bundling image optimization function...
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Downloading https://github.com/stechstudio/libvips-lambda/releases/download/v1.0.1/libvips-8.5.5-lambda.tar.gz/v8.14.4/libvips-8.14.4-linux-arm64v8.tar.br
npm ERR! sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: Prebuilt libvips 8.14.4 binaries are not yet available for linux-arm64v8
npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-23T15_51_06_478Z-debug-0.log
node:internal/errors:865
const err = new Error(message);
^
Error: Command failed: npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" [email protected]
at checkExecSyncError (node:child_process:890:11)
at Object.execSync (node:child_process:962:15)
at createImageOptimizationBundle (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:231:8)
at build (file:///home/john/.npm/_npx/ab9ffda21675e838/node_modules/open-next/dist/build.js:33:5)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
status: 1,
signal: null,
output: [ null, null, null ],
pid: 29241,
stdout: null,
stderr: null
}
Node.js v18.18.2
Error: There was a problem building the "Site" site.
Need help with this error? Post it in #help on the SST Discord https://sst.dev/discord
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
here is the log for the failling sub command on Node.js v18.18.2
john@JOHN:~/workspace/app$ npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/john/workspace/app/apps/web/.open-next/image-optimization-function" [email protected]
npm ERR! code 1
npm ERR! path /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Detected globally-installed libvips v8.14.4
npm ERR! sharp: Building from source via node-gyp
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | x64
npm ERR! gyp info find Python using Python version 3.8.10 found at "/usr/bin/python3"
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Invalid version number: 18
npm ERR! gyp ERR! stack at getNodeDir (/home/john/.nvm/versions/node/v18.18.2/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:57:25)
npm ERR! gyp ERR! stack at /home/john/.nvm/versions/node/v18.18.2/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:31:7
npm ERR! gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
npm ERR! gyp ERR! System Linux 5.15.90.1-microsoft-standard-WSL2
npm ERR! gyp ERR! command "/home/john/.nvm/versions/node/v18.18.2/bin/node" "/home/john/.nvm/versions/node/v18.18.2/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/john/workspace/app/apps/web/.open-next/image-optimization-function/node_modules/sharp
npm ERR! gyp ERR! node -v v18.18.2
npm ERR! gyp ERR! node-gyp -v v9.4.0
npm ERR! gyp ERR! not ok
npm ERR! A complete log of this run can be found in: /home/john/.npm/_logs/2023-10-23T15_51_46_398Z-debug-0.log
when I replace "18" by "18.0.0" the command runs well.
the change is needed just here
npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="${nodeOutputPath}" sharp@${sharpVersion},
npm ERR! sharp: Downloading https://github.com/stechstudio/libvips-lambda/releases/download/v1.0.1/libvips-8.5.5-lambda.tar.gz/v8.14.4/libvips-8.14.4-linux-arm64v8.tar.br
npm ERR! sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: Prebuilt libvips 8.14.4 binaries are not yet available for linux-arm64v8
This seems like sharp-related thing. Can you try installing libvips yourself instead of relying on Sharp? Check out https://formulae.brew.sh/formula/vips. I would encourage you to open issue on sharp's repo, it seems like they are using libvips version which was not pre-compiled for ARM :/
I'm on Windows WSL 2 Ubuntu (x86_64), I've installed brew install vips successfully but I get the exact same error.
It seems like an arm64 compiled libvips is needed to package the lambda.
Am I the only one not being able to deploy my nextjs app on a standard x86 linux system ?
Is everyone using open-next on an arm64 M1 macbook thus being able to get the arm64 libvips straight from brew cmd ?
thank you for your time
Well, it seems WSL-related I think. Either way, I would recommend changing the --arch flag to x86 and subsequently change architecture of image optimiser in CDK.
@khuezy would it make sense to pre-package whole lambda code so it's just copy-paste? I went with that approach in https://github.com/sladg/imaginex-lambda/tree/master as it was too much headache to deal with different OS-es and settings, so prepackaging everything and uploading ready code to Lambda was easiest.
Well, it seems WSL-related I think. Either way, I would recommend changing the
--archflag to x86 and subsequently change architecture of image optimiser in CDK.@khuezy would it make sense to pre-package whole lambda code so it's just copy-paste? I went with that approach in https://github.com/sladg/imaginex-lambda/tree/master as it was too much headache to deal with different OS-es and settings, so prepackaging everything and uploading ready code to Lambda was easiest.
"replacement of shitty build-in NextJS image optimizer." lol
Yea, let's try to use your imagex-lambda. Btw, how shitty is the NextJS image optimizer compared to the python one?
I've found a quick working workaround for the time being
instead of fetching the missing arm libvips from default stechstudio/libvips-lambda
I've been able to override the source to the more richer lovell/sharp-libvips repo
using the npm flag sharp_libvips_binary_host
like this:
$ npm install --arch=arm64 --platform=linux --target=18 --libc=glibc --prefix="/home/nemesis/workspace/app/apps/web/.open-next/image-optimization-function" [email protected] --sharp_libvips_binary_host=https://github.com/lovell/sharp-libvips/releases/download/
added 51 packages in 7s
10 packages are looking for funding
run `npm fund` for details
that could be a quicker fix for open-next
@khuezy it's okay-ish, but there is overhead of starting up Next on image request. Also, trying to separate it efficiently and/or handle dependencies is rather painful. The API is rather simple so recreating in Python with minimal overhead seemed like win-win.
Since the process is async and result is cached at CDN, the extra latency shouldn't be a big deal. edit oh I see, the existing solution has the extra latency due to the next dependency.