🐛 v3 crashes on iOS and macOS devices due to increasing memory usage

[GaLambert]

System Info

• transformers.js version: 3.2.2
• iOS: 18.3.2
• macOS: 15.3.1
• Framework: Next.js (14.2.22) and Vite (6.0.5)

Environment/Platform

• [x] Website/web-app
• [ ] Browser extension
• [ ] Server-side (e.g., Node.js, Deno, Bun)
• [ ] Desktop app (e.g., Electron)
• [ ] Other (e.g., VSCode extension)

Description

We've encountered issues with the v3 of transformers.js (specifically v3.2.2, potentially others) on iOS devices and macOS. The application crashes on iOS (both Safari and Chrome), and causes extremely high and growing memory usage (10+GB) on macOS, eventually also crashing.

Tested with the following examples:

• https://github.com/huggingface/transformers.js-examples/tree/main/next-client
• https://github.com/huggingface/transformers.js-examples/tree/main/video-object-detection

Temporary solution: As a workaround, downgrading to @xenova/[email protected] resolves the iOS crashing issue. However, this is not an ideal long-term solution.

We would greatly appreciate any insights or assistance in resolving these issues with the latest version. Thank you! 🙏

Reproduction

1. Use transformers.js version 3.2.2
2. Run a model (e.g., YoloV9) on an iOS device (iPhone)
3. Observe that the application crashes
4. Run the same model on macOS
5. Check Activity Monitor and observe memory usage exceeding 10GB+ for Safari

[xenova]

Thanks for the report. Could you try again with the latest version to see if this is still an issue? Currently v3.4.0

cc @fs-eire @guschmue too

[Jerboas86]

Thanks for the report. Could you try again with the latest version to see if this is still an issue? Currently v3.4.0

the bug is still there in v3.4.0

I also tried to use models with single threaded onnx runtime, the application still crashes.

[GaLambert]

After testing, I confirm that the bug persists in transformers.js v3.4.0.

[fs-eire]

This may be caused by JSC not doing well with multi-threaded WASM. Could you help to check when you use @xenova/[email protected] and set numThreads to 4, what will the memory usage be like?

[Jerboas86]

With @huggingface/transformers v3.4 on ipadOs 18.3.2, using:

env.backends.onnx.wasm.numThreads = N;

• N = 1 -> Crash / 3.5Go memory usage
• N = 4 -> Crash / 3.5Go memory usage

[fs-eire]

I did a local WASM build for disabling multi-thread support. Could you please try it with transformers.js v3.4.0 with the following setup:

import { env } from "@huggingface/transformers";
env.backends.onnx.wasm.numThreads = 1;
env.backends.onnx.wasm.wasmPaths = {
  mjs: "<path-to>/ort-wasm-simd.mjs",
  wasm: "<path-to>/ort-wasm-simd.wasm",
}

ort-wasm-simd_20250319.zip

EDIT: it would be great helpful if you could help to test the memory usage data for using A, B and C.

env.backends.onnx.wasm.numThreads = 1;
env.backends.onnx.wasm.wasmPaths = {

  // A
  //mjs: '<path-to>/ort-wasm-simd.mjs',
  //wasm: '<path-to>/ort-wasm-simd.wasm',

  // B
  mjs: 'https://cdn.jsdelivr.net/npm/[email protected]/dist/ort-wasm-simd-threaded.mjs',
  wasm: 'https://cdn.jsdelivr.net/npm/[email protected]/dist/ort-wasm-simd-threaded.wasm'

  // C
  //mjs: 'https://cdn.jsdelivr.net/npm/[email protected]/dist/ort-wasm-simd-threaded.jsep.mjs',
  //wasm: 'https://cdn.jsdelivr.net/npm/[email protected]/dist/ort-wasm-simd-threaded.jsep.wasm'
};

[GaLambert]

In my setup: On installing v3.4.0, and debugging the env, the env.backends is undefined on starting my worker that initialises the model and makes inferences. I can add env.backends.onnx.numThreads = 1, and indicate the local .mjs and .wasm files though, but it doesn't influence the application

https://cdn.jsdelivr.net/npm/@huggingface/[email protected]/dist/ort-wasm-simd-threaded.jsep.wasm is automatically installed on starting the worker.

Where the memory goes up again to 10+GB on macOS Safari

[fs-eire]

Could you please try this branch:

https://github.com/fs-eire/transformers.js-examples/tree/investigate-issue-1242

It works on both of my windows and mac but I didn't reproduce the memory issue on both.

[GaLambert]

That indeed fixes the issue @fs-eire . Thanks!

[fs-eire]

That indeed fixes the issue @fs-eire . Thanks!

Thank you for the test. Which option fixes the issue, A or B?

[GaLambert]

I tested A (with the 2 files from the example repo) and B. Both solve the issue.

[Jerboas86]

I confirm that case A works fine on ipadOs 18.3.2. I applied the fix using ort zip files, following these instructions:

import { env } from "@huggingface/transformers";

env.backends.onnx.wasm.numThreads = 1;
env.backends.onnx.wasm.wasmPaths = {
  mjs: "<path-to>/ort-wasm-simd.mjs",
  wasm: "<path-to>/ort-wasm-simd.wasm",
}

[fs-eire]

I confirm that case A works fine on ipadOs 18.3.2. I applied the fix using ort zip files, following these instructions:

import { env } from "@huggingface/transformers";

env.backends.onnx.wasm.numThreads = 1; env.backends.onnx.wasm.wasmPaths = { mjs: "/ort-wasm-simd.mjs", wasm: "/ort-wasm-simd.wasm", }

could you please also help to check using ort-wasm-simd-threaded ?

If ort-wasm-simd-threaded.jsep does not work but ort-wasm-simd-threaded works, it indicates that the problem is ASYNCIFY build. If both above don't work, only ort-wasm-simd works, it indicates the problem is in MT support of JSC.

[Jerboas86]

could you please also help to check using ort-wasm-simd-threaded ?

Case A ✅ Case B ✅ case C ❌

[Jerboas86]

Since Ipados v18.4, the bug is fixed at leat for now See bug webkit report, using transformers.js v3.5.1

[fs-eire]

Since Ipados v18.4, the bug is fixed at leat for now See bug webkit report, using transformers.js v3.5.1

Thank you for the information! It’s still unknown whether the original issue is fixed in v18.4 because transformerjs did a workaround in 3.5.1.

Is it working in 3.4.0 on iOS 18.4?

[Jerboas86]

Is it working in 3.4.0 on [ipad]OS 18.4 ?

Yes

[PierreMesure]

Can confirm it is fixed with 3.3.3 which I use right now for whisper-web.mesu.re.

[fs-eire]

thank you for the confirmation.

so it looks like the root cause is fixed in iOS 18.4

Before Safari released with Webgpu support by default, we can keep the workaround.

[xuwenguang]

hmm whisperBase.en still fails on my iphone with transformer.js 3.5.1 & IOS 18.4. Is the fix released in 3.5.1 @xenova ?

[PierreMesure]

Can you confirm that you are not getting issue #1273 @xuwenguang? For me, any version after 3.3.3 is broken for Whisper models.

[xuwenguang]

@PierreMesure any version above 3.3.3 is still failing for me. Even in 3.3.3 bigger models like quantized whisper small still crashes my app...

But I see it works in your demo site. Are you using the wasm file shared in this thread or just using the default setting? https://github.com/huggingface/transformers.js/issues/1242#issuecomment-2738598654

[PierreMesure]

No modified files, just vanilla transformers.js 3.3.3 and the same code that @xenova wrote for his demo. So in my case, iOS 18.4 solved the issue. You can have a look at the source code here.