transformers.js icon indicating copy to clipboard operation
transformers.js copied to clipboard
verified publisher icon xenova

Next.js server example is broken

Open nguyenmp opened this issue 1 year ago • 8 comments

System Info

node -v
v22.3.0

git show -s
commit 7f5081da29c3f77ee830269ab801344776e61bcb (HEAD -> main, origin/main, origin/HEAD)
Author: Joshua Lochner <[email protected]>
Date:   Tue Jul 2 11:15:38 2024 +0200

    Update example projects dependencies (#834)
    
    * Update example project dependencies
    
    * Update jinja version

MacOS 14.5 (23F79)

Environment/Platform

  • [X] Website/web-app
  • [ ] Browser extension
  • [X] Server-side (e.g., Node.js, Deno, Bun)
  • [ ] Desktop app (e.g., Electron)
  • [ ] Other (e.g., VSCode extension)

Description

When I try to run the example server-side Next.js app, it fails in two ways:

#440 seems to have an accepted solution that isn't patched yet.

#741 I think my solution is novel, no one else has posted about it as far as I can tell. For this one, it seems like when we do the normal import * from 'onnxruntime-node', it returns an empty module for some reason. Is it possible there's some circular dependency? When I replace it with const onnx_node = require('onnxruntime-node');, everything works out great!

Here's a patch that fixes my issues:

diff --git a/examples/next-server/next.config.js b/examples/next-server/next.config.js
index a1a4c56..580966d 100644
--- a/examples/next-server/next.config.js
+++ b/examples/next-server/next.config.js
@@ -1,13 +1,20 @@
 /** @type {import('next').NextConfig} */
 const nextConfig = {
-    // (Optional) Export as a standalone site
-    // See https://nextjs.org/docs/pages/api-reference/next-config-js/output#automatically-copying-traced-files
-    output: 'standalone', // Feel free to modify/remove this option
-    
     // Indicate that these packages should not be bundled by webpack
     experimental: {
         serverComponentsExternalPackages: ['sharp', 'onnxruntime-node'],
     },
+
+    // Override the default webpack configuration
+    webpack: (config) => {
+        // See https://webpack.js.org/configuration/resolve/#resolvealias
+        config.resolve.alias = {
+            ...config.resolve.alias,
+            "sharp": false,
+            "onnxruntime-node": false,
+        }
+        return config;
+    },
 };
 
 module.exports = nextConfig;
diff --git a/src/backends/onnx.js b/src/backends/onnx.js
index 0bee3dce7184d1fec1bd0e75677a6fea3a47db30..834fad5b3720dda85670aa5e79329f0ac71dc9bb 100644
--- a/src/backends/onnx.js
+++ b/src/backends/onnx.js
@@ -31,7 +31,8 @@ export const executionProviders = [
 
 if (typeof process !== 'undefined' && process?.release?.name === 'node') {
     // Running in a node-like environment.
-    ONNX = ONNX_NODE.default ?? ONNX_NODE;
+    const onnx_node = require('onnxruntime-node');
+    ONNX = onnx_node.default ?? onnx_node;
 
     // Add `cpu` execution provider, with higher precedence that `wasm`.
     executionProviders.unshift('cpu');

Reproduction

Run (server-side inference) https://github.com/xenova/transformers.js/tree/main/examples/next-server from top of tree.

nguyenmp avatar Jul 30 '24 20:07 nguyenmp

You didn't mention it, but I saw the pnpm lock file in your commit and I'm pretty sure this is just another case of pnpm being a bad replacement - the example works like a charm on default npm.

Replacing ESM with require also feels like two steps backwards - make sure the tools you use work properly in 2024.

kungfooman avatar Jul 31 '24 14:07 kungfooman

using node v18.17.0 nextjs 14.2.5 "@xenova/transformers": "^2.17.2",

and with npm

I am on a M1 mac (not sure if that has anything to do with anything)

getting the same failure of

 ⨯ ./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)

Import trace for requested module:
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/ sync ^\.\/sharp\-.*\.node$
./node_modules/@xenova/transformers/node_modules/sharp/lib/sharp.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/constructor.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/index.js
./node_modules/@xenova/transformers/src/utils/image.js
./node_modules/@xenova/transformers/src/transformers.js
./src/app/api/test/xenova.ts
./src/app/api/test/route.ts
 ⨯ ./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)

Import trace for requested module:
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/ sync ^\.\/sharp\-.*\.node$
./node_modules/@xenova/transformers/node_modules/sharp/lib/sharp.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/constructor.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/index.js
./node_modules/@xenova/transformers/src/utils/image.js
./node_modules/@xenova/transformers/src/transformers.js
./src/app/api/test/xenova.ts
./src/app/api/test/route.ts

rosman21 avatar Aug 06 '24 21:08 rosman21

Maybe solution for pnpm also works for npm?

https://github.com/xenova/transformers.js/issues/210#issuecomment-2241539362

@desaxce Do you know more, since you just posted your solution?

kungfooman avatar Aug 07 '24 07:08 kungfooman

Maybe solution for pnpm also works for npm?

#210 (comment)

@desaxce Do you know more, since you just posted your solution?

I tried that and didn’t work for me so I switched to npm and still had issues

using node v18.17.0 nextjs 14.2.5 "@xenova/transformers": "^2.17.2",

and with npm

I am on a M1 mac (not sure if that has anything to do with anything)

getting the same failure of

 ⨯ ./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)

Import trace for requested module:
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/ sync ^\.\/sharp\-.*\.node$
./node_modules/@xenova/transformers/node_modules/sharp/lib/sharp.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/constructor.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/index.js
./node_modules/@xenova/transformers/src/utils/image.js
./node_modules/@xenova/transformers/src/transformers.js
./src/app/api/test/xenova.ts
./src/app/api/test/route.ts
 ⨯ ./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)

Import trace for requested module:
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/sharp-darwin-arm64v8.node
./node_modules/@xenova/transformers/node_modules/sharp/build/Release/ sync ^\.\/sharp\-.*\.node$
./node_modules/@xenova/transformers/node_modules/sharp/lib/sharp.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/constructor.js
./node_modules/@xenova/transformers/node_modules/sharp/lib/index.js
./node_modules/@xenova/transformers/src/utils/image.js
./node_modules/@xenova/transformers/src/transformers.js
./src/app/api/test/xenova.ts
./src/app/api/test/route.ts

My version information

rosman21 avatar Aug 07 '24 13:08 rosman21

Y'all are absolutely right. npm i followed by npm run dev works fine. In my case, I was accidentally using both client side and server side at the same time, which I did manage to get working but not without hacks.

nguyenmp avatar Aug 07 '24 20:08 nguyenmp

hey @nguyenmp any tips on getting this to work on both client and server?

Using npm hasn't solved my issue.

TypeError: Cannot read properties of undefined (reading 'create') at constructSession (webpack-internal:///(rsc)/./node_modules/.pnpm/@[email protected]/node_modules/@xenova/transformers/src/models.js:468:39)

My client side code is working, but this error persists when trying to incorporate the server side version.

derfcode avatar Sep 05 '24 10:09 derfcode

Can you try https://github.com/huggingface/transformers.js-examples/tree/main/next-server to see if it works for your use-case?

Demo: https://huggingface.co/spaces/webml-community/next-server-template

Solution:

const nextConfig = {
  output: "standalone",
  // https://nextjs.org/docs/app/api-reference/next-config-js/serverExternalPackages
  serverExternalPackages: ["@huggingface/transformers"],
};

xenova avatar Oct 23 '24 05:10 xenova 

const nextConfig = {
  serverExternalPackages: ["@huggingface/transformers"]
};

this was fixing it for me in a nextjs 15.x + turbopack project.

TimPietrusky avatar Feb 15 '25 17:02 TimPietrusky

Has anyone managed to deploy to vercel's api routes?

Zefty avatar Sep 14 '25 22:09 Zefty

Closed by https://github.com/huggingface/transformers.js/issues/875#issuecomment-2430914589

xenova avatar Oct 13 '25 04:10 xenova