"Can't create a session.failded to allocate a buffer of size 23200850"

[rockmany2000]

System Info

Transformer.js 3.5.1 , Chrome in Android Phone

Environment/Platform

• [x] Website/web-app
• [ ] Browser extension
• [ ] Server-side (e.g., Node.js, Deno, Bun)
• [ ] Desktop app (e.g., Electron)
• [ ] Other (e.g., VSCode extension)

Description

Transformer.js 3.5.1 , after "transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-base');"

1. Chrome 132.0.6834.165 , realme Q3, 6G+128G. it will fail with "Can't create a session.failded to allocate a buffer of size 23200850"
2. Chrome 132.0.6834.123 , Samsung S20+ ,8G+64G, it works ok.
3. Chrome 136.0.7103.87 , PH-1 , 3G+128G, it works ok. I make a memory tester - for 1, It only allocate about 300-500MB. for 2,3 ,it can allocate about 4GB. function testMaxMemory() { let pages =100; // 62.5MB try { while (true) { const memory = new WebAssembly.Memory({ initial: pages, maximum: pages }); console.log( ${pages * 65536 / (1024 * 1024)} MB); pages += 100;
   } } catch (e) { console.error( ${e.message}); console.log( ${(pages - 100) * 65536 / (1024 * 1024)} MB); } } After searching online, I believe that this problem is related to the memory allocation of Android to WASM, but there is currently a lack of a solution, which makes it impossible to reliably deploy Transformers.js on mobile phones.

https://github.com/WebAssembly/design/issues/1397

Reproduction

transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-base');

[xenova]

Hi there 👋 A buffer size of 23200850 bytes is pretty small (~23 MB), but this seems to be an issue specific to Android.

Can you try with https://huggingface.co/onnx-community/whisper-base? That version should be able to reduce memory footprint slightly.

[SignOfZeta]

Hi there 👋 A buffer size of 23200850 bytes is pretty small (~23 MB), but this seems to be an issue specific to Android.

Can you try with https://huggingface.co/onnx-community/whisper-base? That version should be able to reduce memory footprint slightly.

I have faced a similar issue. I tried loading tiny and base on an android device with 8GB LPDDR4X, both loads but goes OOM pretty quick after a few seconds of transcription.

@rockmany2000 ~~Latest stable Chrome and chromium-based Samsung Internet Browser from Play Store only allocates at most 1026MB for me. How did you get to 4GB? Are you using a modified build of chromium?~~ Update: I ran your memory growing script and performance.memory.jsHeapSizeLimit together.

<script>
let pages = 100;
try {
    while (true) {
        const memory = new WebAssembly.Memory({
            initial: pages,
            maximum: pages
        });
        pages += 100
		
		allocatedMemory = `${(pages-100) * 65536 / (1024*1024)} MB`;
		wasmMemoryLimit = '4096 MB';
		usedHeapSize = `${Math.floor(performance.memory.usedJSHeapSize / 1024 / 1024)} MB`;
		browserHeapSizeLimit = `${Math.floor(performance.memory.jsHeapSizeLimit / 1024 / 1024)} MB RAM`;
		
		console.log(`Allocated: ${allocatedMemory}/${wasmMemoryLimit}\nHeap: ${usedHeapSize}/${browserHeapSizeLimit}`);
    }
} catch (e) {
    console.error(`${e.message}`);
	
	allocatedMemory = `${(pages-100) * 65536 / (1024*1024)} MB`;
	wasmMemoryLimit = '4096 MB';
	usedHeapSize = `${Math.floor(performance.memory.usedJSHeapSize / 1024 / 1024)} MB`;
	browserHeapSizeLimit = `${Math.floor(performance.memory.jsHeapSizeLimit / 1024 / 1024)} MB RAM`;
    
	window.prompt(`Allocated: ${allocatedMemory}/${wasmMemoryLimit}\nHeap: ${usedHeapSize}/${browserHeapSizeLimit}`);
}
</script>

On windows x86 chromium latest stable, it shows

On android chromium latest stable, it shows

Interestingly, I also came across this issue that Unify folks encountered a few years ago, and at that time android chromium WASM was only able to allocate 256MB-512MB RAM. Unity forums Chromium issues

This could be the reason of this issue, maybe WASM doesn't actually succeed in allocating that large amount of memory beyond max heap.

[rockmany2000]

Yes, I encountered the following problems:

1. Some Android phones have memory limitations when using WASM, which causes Transformers.js to fail to load the model.
2. I cannot know the size of the free memory space of WASM, so I can only try to allocate space for testing, which may sometimes cause the browser to exit abnormally.
3. I cannot change the limit of WASM memory size.

[rockmany2000]

Still not working..

Hi there 👋 A buffer size of 23200850 bytes is pretty small (~23 MB), but this seems to be an issue specific to Android.

Can you try with https://huggingface.co/onnx-community/whisper-base? That version should be able to reduce memory footprint slightly.

[SignOfZeta]

Still not working..

Hi there 👋 A buffer size of 23200850 bytes is pretty small (~23 MB), but this seems to be an issue specific to Android. Can you try with https://huggingface.co/onnx-community/whisper-base? That version should be able to reduce memory footprint slightly.

Have you tried whisper-tiny ONNX? If still not working, it could be some device settings I guess? I would start by fiddling with some settings via ADB (or if you have root access on that device then via a terminal emulator).

[xenova]

Still not working..

Since this is sounding like a device limitation, it seems like the best solution for now is to choose a slightly smaller model for those devices, like https://huggingface.co/onnx-community/whisper-tiny.

[rockmany2000]

Still not working..

Since this is sounding like a device limitation, it seems like the best solution for now is to choose a slightly smaller model for those devices, like https://huggingface.co/onnx-community/whisper-tiny.

On this phone, the results are very unstable, sometimes the model can be loaded and sometimes it works, but sometimes it will OOM.

An error occurred during model execution: "RangeError: Array buffer allocation failed"

[SignOfZeta]

Still not working..

Since this is sounding like a device limitation, it seems like the best solution for now is to choose a slightly smaller model for those devices, like https://huggingface.co/onnx-community/whisper-tiny.

On this phone, the results are very unstable, sometimes the model can be loaded and sometimes it works, but sometimes it will OOM.

An error occurred during model execution: "RangeError: Array buffer allocation failed"

Can you show me your chrome://flags enabled settings? I suspect the unsafe/experimental WebGPU features flag leads to different results on different mobile SoC.