[Web] Previously used .wasm binaries missing in 1.19.0 (using web bundler with CopyPlugin)

[e-simpson]

Describe the issue

Using a onnxruntime-web with a web bundler like CopyPlugin I am now getting Package path ./webgpu is not exported from package. When I manually try to bundle the .wasm's I get Unable to locate '..node_modules/onnxruntime-web/dist/ort-wasm.wasm' glob.

I assume the names changed with this update as I see the web bundling changed. I see now bundling was introduced with https://github.com/microsoft/onnxruntime/commit/dd805ff77d896e9ff5457818c49d2103d6ba6248. I looked for some migration docs but couldn't find any.

To reproduce

Can follow the current quick-start https://github.com/microsoft/onnxruntime-inference-examples/tree/main/js/quick-start_onnxruntime-web-bundler

or

1. Use next.js or similar web bundler/pack framework
2. Use webpack copy plugin to load onnxruntime-web. Example for next.js:

/** @type {import('next').NextConfig} */
import CopyPlugin from "copy-webpack-plugin";

const nextConfig = {
  webpack: (config) => {
    config.resolve.extensions.push(".ts", ".tsx");
    config.resolve.fallback = { fs: false };

    config.plugins.push(
      new CopyPlugin({
        patterns: [
          {
            from: './node_modules/onnxruntime-web/dist/ort-wasm.wasm',
            to: 'static/chunks/pages',
          },
          {
            from: './node_modules/onnxruntime-web/dist/ort-wasm-simd.wasm',
            to: 'static/chunks/pages',
          },
          { 
            from: './node_modules/onnxruntime-web/dist/ort-wasm-simd.jsep.wasm',
            to: 'static/chunks/pages',
          },
          ],
        }),
    );

    return config;
  } 
}

export default nextConfig;

3. Import onnxruntime-web/webgpu with import * as ort from 'onnxruntime-web/webgpu';
4. Run the dev server, example: bun run dev

I get the error:

Module not found: Package path ./webgpu is not exported from package .../node_modules/onnxruntime-web

Urgency

Not urgent, rolled back to 1.18.0 for now.

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.0

Execution Provider

'webgpu' (WebGPU)

[gotomypc]

The solution to fix this, can be npm install [email protected], all fix the package.json, lock the onnxruntime-web to 1.14.0 version.

[e-simpson]

@gotomypc I use 1.18.0 currently which works great. I was just looking to use some new features from 1.19.0😊

[fs-eire]

There are multiple changes for onnxruntime-web exports.

a TL;DR version:

• if you use WebGPU, then
  • import * as ort from 'onnxruntime-web/webgpu';
  • copy ort-wasm-simd-threaded.jsep.wasm
• if you use CPU only, then
  • import * as ort from 'onnxruntime-web';
  • copy ort-wasm-simd-threaded.wasm

Just 2 .wasm files. copy the one you uses. See this link to check out options for importing.

Let me know if you have further questions

[fs-eire]

work item: examples should be updated.

[e-simpson]

@fs-eire thanks for the reply!

I copied the ort-wasm-simd-threaded.jsep.wasm into my static files, made sure I'm using 1.19.0 and importing with import * as ort from 'onnxruntime-web/webgpu'; I get the error:

Module not found: Package path ./webgpu is not exported from package 
.../node_modules/onnxruntime-web (see exports field in .../node_modules/onnxruntime-web/package.json)
> 1 | import * as ort from 'onnxruntime-web/webgpu';
    | ^
  2 |

Strange. I tried to copy the ort-wasm-simd-threaded.wasm and use the base non-gpu version and nothing seems to be working. If I revert back to 1.18.0 everything seems to work.

[fs-eire]

@fs-eire thanks for the reply!

I copied the ort-wasm-simd-threaded.jsep.wasm into my static files, made sure I'm using 1.19.0 and importing with import * as ort from 'onnxruntime-web/webgpu'; I get the error:

Module not found: Package path ./webgpu is not exported from package 
.../node_modules/onnxruntime-web (see exports field in .../node_modules/onnxruntime-web/package.json)
> 1 | import * as ort from 'onnxruntime-web/webgpu';
    | ^
  2 |

Strange. I tried to copy the ort-wasm-simd-threaded.wasm and use the base non-gpu version and nothing seems to be working. If I revert back to 1.18.0 everything seems to work.

What bundler (and version) do you use? It looks like that your bundler is not reading from the "export" field of onnxruntime-web's package.json

[e-simpson]

@fs-eire Bundling with webpack 5.93.0 (what the latest Next.js comes with)

Yeah, and I see the 1.19.0 package exports for web-gpu

    "./webgpu": {
      "node": null,
      "import": "./dist/ort.webgpu.bundle.min.mjs",
      "require": "./dist/ort.webgpu.min.js",
      "types": "./types.d.ts"
    },

For reference the web-gpu exports for 1.18.0

    "./webgpu": {
      "import": "./dist/esm/ort.webgpu.min.js",
      "require": "./dist/cjs/ort.webgpu.min.js",
      "types": "./types.d.ts",
      "default": "./dist/ort.webgpu.min.js"
    },

Let me give turbopack a try

[fs-eire]

Since it's Next.js, make sure not using SSR (because importing 'onnxruntime-web/webgpu' from Node.js is explicitly disabled)

[illbexyz]

I'm having the same issue. I made sure to load onnxruntime-web/webgpu dynamically to filter out SSR errors, but the error seems to be triggered at build time.

I made a reproduction here: https://github.com/illbexyz/nextjs-onnxruntime. See https://github.com/illbexyz/nextjs-onnxruntime/blob/main/src/pages/index.tsx where I import <OnnxButton /> (which contains the import to onnxruntime-web/webgpu) dynamically with ssr: false.

[fs-eire]

I'm having the same issue. I made sure to load onnxruntime-web/webgpu dynamically to filter out SSR errors, but the error seems to be triggered at build time.

I made a reproduction here: https://github.com/illbexyz/nextjs-onnxruntime. See https://github.com/illbexyz/nextjs-onnxruntime/blob/main/src/pages/index.tsx where I import <OnnxButton /> (which contains the import to onnxruntime-web/webgpu) dynamically with ssr: false.

I cloned the repo locally and I can reproduce this error. I am not sure how Next.js deals with this dynamic loading. But I have confirmed that even with ssr: false, when Next.js parses the line:

import * as onnx from "onnxruntime-web";

in OnnxButton.tsx, it uses the "node" export.

This is confirmed by replace the following in node_modules/onnxruntime-web/package.json:

  "exports": {
    ".": {
-     "node": {
-       "import": "./dist/ort.node.min.mjs",
-       "require": "./dist/ort.node.min.js"
-     },
+     "node": null,
      "import": "./dist/ort.bundle.min.mjs",
      "require": "./dist/ort.min.js",
      "types": "./types.d.ts"
    },

This will generate the following error message:

Module not found: Package path . is not exported from package C:\code\nextjs-onnxruntime\node_modules\onnxruntime-web

Conclusion:

The error is because Next.js module resolver picks export for "node" not "import"/"require"(web) in your repo. I don't know exactly how to change this behavior, but if Next.js picks the "import"/"require"(web) export, the issue should be fixed.

[Systemcluster]

I made a reproduction here: illbexyz/nextjs-onnxruntime.

It's missing onnxruntime-web in the dependencies, and both next dev and next build work for me after adding it.

[fs-eire]

I made a reproduction here: illbexyz/nextjs-onnxruntime.

It's missing onnxruntime-web in the dependencies, and both next dev and next build work for me after adding it.

Yes, you are right. It's missing in the dependencies and need to run npm install onnxruntime-web@dev. But I think @illbexyz is complaining about cannot import from "onnxruntime-web/webgpu" (while the repo uses "onnxruntime-web" and it works, modifying the line in OnnxButton.tsx will show that error)

[illbexyz]

Yes sorry my bad, I provided a quite sloppy reproduction 😅 In case it might be useful to someone, I updated the reproduction to:

• declare onnxruntime-web@dev in package.json
• use onnxruntime-web/webgpu in OnnxButton.tsx

The error is because Next.js module resolver picks export for "node" not "import"/"require"(web) in your repo. I don't know exactly how to change this behavior, but if Next.js picks the "import"/"require"(web) export, the issue should be fixed.

Yes I agree with this conclusion, but I have no idea on how to configure next.js to pick the "import"/"require"(web) export.

[e-simpson]

Yeah, there has to be a "next.js" way to get around this either with some aliases, dynamic resolving, or client flags.

However, for now, I've just worked around it and have been using 1.19.x just as I did before in next.js by manually removing the "node": null, lines from module's package.json. My onnxruntime-web/package.json now looks like:

...
"exports": {
    ".": {
      "node": {
        "import": "./dist/ort.node.min.mjs",
        "require": "./dist/ort.node.min.js"
      },
      "import": "./dist/ort.bundle.min.mjs",
      "require": "./dist/ort.min.js",
      "types": "./types.d.ts"
    },
    "./all": {
      "import": "./dist/ort.all.bundle.min.mjs",
      "require": "./dist/ort.all.min.js",
      "types": "./types.d.ts"
    },
    "./wasm": {
      "import": "./dist/ort.wasm.min.mjs",
      "require": "./dist/ort.wasm.min.js",
      "types": "./types.d.ts"
    },
    "./webgl": {
      "import": "./dist/ort.webgl.min.mjs",
      "require": "./dist/ort.webgl.min.js",
      "types": "./types.d.ts"
    },
    "./webgpu": {
      "import": "./dist/ort.webgpu.bundle.min.mjs",
      "require": "./dist/ort.webgpu.min.js",
      "types": "./types.d.ts"
    },
    "./training": {
      "import": "./dist/ort.training.wasm.min.mjs",
      "require": "./dist/ort.training.wasm.min.js",
      "types": "./types.d.ts"
    }
  },
  ...

And everything works as before.

[fs-eire]

the original idea of adding "node": null, is to avoid misuse in Node.js. It seems by removing them indeed helps to use onnxruntime-web in Next.js... if not using web worker. (#22113 is the issue of next step if trying to enable web worker)

[fs-eire]

@illbexyz I verified the solution mentioned in https://github.com/vercel/next.js/issues/70296#issuecomment-2365441435 works.

Fix: https://github.com/fs-eire/nextjs-onnxruntime/commit/b7adf6f0367c81532e5173206f44e0f06da30bbe

• use 'use client'; at the top of the CSR component
• use dynamic import and use if (typeof windows !== 'undefined') to wrap the dynamic import so that it will not be compiled as Node.js usage.

[illbexyz]

@fs-eire Thank you! I can confirm that the solution works correctly on Next 14. Unfortunately it doesn't work on Next <= 12 where 'use client' wasn't introduced yet, but imho it's not a big deal.

If @e-simpson agrees I think we can close the issue 🙂

[e-simpson]

Yes let's consider this solved 🥳

[blueanchovy]

for anyone using @imgly and getting this issue, nextjs 14.2.8 with @imgly/background-removal 1.5.5 works for me. trying the above solutions didn't really solve the issue for me so putting it out here