I am getting no available backend found error in a browser environment

[Nithur-M]

System Info

"@huggingface/transformers": "^3.3.1",

Chrome

Environment/Platform

• [ ] Website/web-app
• [x] Browser extension
• [ ] Server-side (e.g., Node.js, Deno, Bun)
• [ ] Desktop app (e.g., Electron)
• [ ] Other (e.g., VSCode extension)

Description

I am getting the following error in a browser extension: Error loading model: Error: no available backend found. ERR: [webgpu] Error: Cannot find module 'https://cdn.jsdelivr.net/npm/@huggingface/[email protected]/dist/ort-wasm-simd-threaded.jsep.mjs'

Please let me know if there is any fix for this.

Reproduction

Install the latest version and try to run it in a browser extension

[qdrk]

Same here, trying to use in the background script. Also see discussion here: https://github.com/microsoft/onnxruntime/pull/20991

[Nithur-M]

I am seeing a different issue than mentioned in the above discussion @qdrk

[qdrk]

I am seeing a different issue than mentioned in the above discussion @qdrk

You are right, this looks different.

[djaffer]

same error. This package is so hard to use.

[elloza]

I got the same error in the case of a browser extension using the Plasmo framework.

I have tried to apply what was discussed in this thread with the PR (https://github.com/microsoft/onnxruntime/pull/20991#issuecomment-2622947140) that @fs-eire mentioned but it still gives the error.

Any help is welcome because right now I don't really know where the error can come from....

I've tried to load that file in assets statically but I don't know how to change the path to load ort-wasm-simd-threaded.jsep.mjs...

Thank you very much!

[fs-eire]

I believe that this PR should fix the issue.

If you can do a local build, please help to check if the fix works to you.

The change is planned to be released in version 3.4

[xenova]

If you can do a local build, please help to check if the fix works to you.

Yes please, if you could try a local build of that PR, that would be great!

Also, @Nithur-M could you try v3.3.3?

FYI: I'm busy wrapping up v3.4 - just working on a few more things.

[grolox69]

Hello @fs-eire @xenova, I have built the PR locally and still having the same issue mentioned by @Nithur-M. Running in a Chrome extension background script.

[fs-eire]

could you share me a repo so that I can try to debug

[grolox69]

@fs-eire https://github.com/grolox69/kokorojs-extension, follow steps 5,6,7 under Installing and Running to setup ext. I am using kokoro.js (uses "@huggingface/transformers": "^3.3.3" ).

STR:

1. Open the extension Popup.
2. Click on the speak button.

[fs-eire]

I checked the repo and I can reproduce the issue. Here are some results of my investigations:

• The dependency chain is kokorojs-extension > kokorojs > transformerjs > onnxruntime-web
• The default configuration of bundler is only working for a "final product", not for a "library". This means in this example, kokorojs, transformerjs and onnxruntime-web need to carefully setup settings for bundlers. While I have spent some effort on making transformerjs and onnxruntime-web correctly work with bundlers, kokorojs is not. There are a few issues that causes kokorojs not supporting a service worker usage.
• There are also several problems in the background.bundle.js. I cannot figure out why, but at least webpack is not correctly setup for a service worker + ESM target for background.bundle.js.

I remember that there is a very simple working example of using service worker which directly using transformerjs. probably we need some document for using transformerjs/onnxruntime-web in a service worker.

[andrespm2000]

I'm also getting a backend error when trying to fetch the models from a Plasmo project.

https://github.com/andrespm2000/basic-text-classifier

Not only is it giving an error with the louisthomaspro/xlm-roberta-base-language-detection-onnx model, but pretty much any other I've tried with.

[grolox69]

@fs-eire @xenova Going back to the working example you mentioned, I guessed it's this extension example.

This example is outdated, and does not work correctly with new version of @huggingface/transformers. This a different error than the one seen in this issue, but relates to extension compatibility.

Error: Error: no available backend found. ERR: [wasm] TypeError: import() is disallowed on ServiceWorkerGlobalScope by the HTML specification.

STR:

1. Clone the example
2. Install latest version of @huggingface/transformers (npm i @huggingface/transformers)
3. Replace @xenova/transformers with @huggingface/transformers in background.js

Plus, can you please explain the difference between the @huggingface/transformers and @xenova/transformers npm packages? @xenova/transformers github repo redirects to @huggingface/transformers repo.

[fs-eire]

@fs-eire @xenova Going back to the working example you mentioned, I guessed it's this extension example.

This example is outdated, and does not work correctly with new version of @huggingface/transformers. This a different error than the one seen in this issue, but relates to extension compatibility.

Error: Error: no available backend found. ERR: [wasm] TypeError: import() is disallowed on ServiceWorkerGlobalScope by the HTML specification.

STR:

1. Clone the example
2. Install latest version of @huggingface/transformers (npm i @huggingface/transformers)
3. Replace @xenova/transformers with @huggingface/transformers in background.js

Plus, can you please explain the difference between the @huggingface/transformers and @xenova/transformers npm packages? @xenova/transformers github repo redirects to @huggingface/transformers repo.

I didn't use that example. Instead, I tried the one that in this discussion: https://github.com/microsoft/onnxruntime/pull/20991#issuecomment-2622947140

[fs-eire]

@grolox69 @xenova I created #1213 to fix the extension example

[xenova]

Thanks @fs-eire! We're in the process of moving the examples to https://github.com/huggingface/transformers.js-examples/tree/main/browser-extension. Will your fix work there too?

[killuday]

// This file explicitly imports the ONNX runtime for Transformers.js
import { env } from '@huggingface/transformers';

export async function setupOnnxRuntime() {
  try {
    // Dynamically import onnxruntime-web
    await import('onnxruntime-web');
    
    // Configure the ONNX backend
    env.backends = {
      ...env.backends,
      onnx: {
        wasm: {
          proxy: true
        }
      }
    };
    
    console.log('ONNX Runtime initialized successfully');
    return true;
  } catch (error) {
    console.error('Failed to initialize ONNX Runtime:', error);
    return false;
  }
} 

worked for me

Use it like this while initializing the model

  // Explicitly setup ONNX runtime
    await setupOnnxRuntime();

[ArvineKwok]

I'm facing the same issue. I use plasmo to build my exetension, I've tried set wasmPaths to "" or absolute path of .mjs file, and add "web_accessible_resources". Even there are .mjs and .wasm files in build directory, but it still said "Error loading model: Error: no available backend found. ERR: [webgpu] Error: Cannot find module ". I'm not sure if this is because plasmo? because in extension example, it works fine with webpack.

[fs-eire]

I am not familiar with plasmo. I think it is using parcel as bundler, so there might be some steps needed to configure parcel to deploy the required files

[ArvineKwok]

I am not familiar with plasmo. I think it is using parcel as bundler, so there might be some steps needed to configure parcel to deploy the required files

Thank you! you inspired me, I will look into it