[Web] 'Worker': ArrayBuffer at index 0 is not detachable and could not be transferred when running inference session

[nezaBacar]

Describe the issue

I load two models cca. 800MB into the browser. When I attempt to run inference using a 650MB model with the ort.env.wasm.proxy flag set to true, the application crashes with an error message: "TypeError: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 0 is not detachable and could not be transferred." Do you have any suggestions for preventing this error? Alternatively, if I don't set the flag to true, the inference process runs smoothly, but it causes the UI to freeze. Are there any alternative methods to prevent the UI from freezing?

To reproduce

• Enable ort.env.wasm.proxy
• Run inference session of this model: https://drive.google.com/drive/folders/1oHPKJZVyeUQiuhtmqMPEHm5ORC4fuPbv?usp=sharing

Urgency

It's somewhat urgent

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.16

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

[fs-eire]

This error means that one ArrayBuffer is used in a detached state. I may need to take a look at your code to figure out the detail.

[nezaBacar]

Thanks for the fast response! I've added you to the demo repository. Just pop the model into the /demo/models folder. And make sure that the line 20 in index.html points to model.onnx and that the line 41 in diffusion.js is not commented. The problem seems to be in lines 91/92 in diffusion.js script. I noticed that if i don't run the model twice the issue does not occur.

[fs-eire]

The root cause if figured out.

When using proxy, input tensors' underlying buffer is transferred from main thread into worker thread to avoid memcpy. This will invalidate any object in main thread who uses that buffer. This is why this error come out because the underlying array buffer in latents is transferred into proxy worker now already.

    let noisePredText = await predict_noise(latents, timestep, positive);    // this works good
    let noisePredUnet = await predict_noise(latents, timestep, negative);    // this cannot work , because 'latents' is based on invalid buffer

I can make a fix to transfer the array buffer back to main thread after using it.

[nezaBacar]

Thanks for your response. Oh I understand, yes please make the fix if possible. I would appreciate it so much.

[fs-eire]

@nezaBacar could you help to verify if the PR fixes the issue by consuming package built here: https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=1306843&view=artifacts&pathAsName=false&type=publishedArtifacts