onnxruntime icon indicating copy to clipboard operation
onnxruntime copied to clipboard

use WebGPU EP instead of JSEP in WebAssembly

Open fs-eire opened this issue 7 months ago • 7 comments

Description

This PR allows to use WebGPU EP in onnxruntime-web NPM package.

Migration Plan

Currently, there are 2 different EPs implemented the WebGPU backend of onnxruntime-web. They are JSEP and WebGPU EP. The migration plan is to replace the JSEP with WebGPU EP and eventually remove the JSEP. The plan contains the following stages:

  • STAGE 1: enable WebGPU EP on onnxruntime-web in local build. (Done)
  • STAGE 2: enable WebGPU EP on onnxruntime-web in the public package. (This PR)
  • STAGE 3: remove JSEP from onnxruntime-web.

Package consumption changes

  • Default import (import 'onnxruntime-web') and CPU only import (import 'onnxruntime-web/wasm') keeps their previous behaviors.

  • WebGPU import (import 'onnxruntime-web/webgpu') will now use WebGPU EP instead of JSEP. Previously it was the same as default import.

    • WebGPU import will use a different suffix for the .mjs and .wasm name (which was .jsep):

      • ort-wasm-simd-threaded.asyncify.mjs
      • ort-wasm-simd-threaded.asyncify.wasm
    • The suffix .asyncify is used as .jspi is planned for future. They are 2 different ways of emscripten's async implementation (sync C++ function calls async JS function)

fs-eire avatar May 25 '25 01:05 fs-eire

TODO: resolve conflicts after #25022 is merged.

fs-eire avatar Jun 11 '25 18:06 fs-eire

both jsep and webgpu-ep builds for for me.

guschmue avatar Jun 17 '25 19:06 guschmue

@Honry could you please help to test/validate if this change breaks WebNN?

This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

fs-eire avatar Jun 17 '25 21:06 fs-eire

@Honry could you please help to test/validate if this change breaks WebNN?

This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

Honry avatar Jun 18 '25 01:06 Honry

@Honry could you please help to test/validate if this change breaks WebNN? This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

@fs-eire, both work! BTW, will we have a build specific for WebNN? e.g. ort.webnn.min.js, onnxruntime-web/webnn

Honry avatar Jun 18 '25 03:06 Honry

@Honry could you please help to test/validate if this change breaks WebNN? This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

@fs-eire, both work! BTW, will we have a build specific for WebNN? e.g. ort.webnn.min.js, onnxruntime-web/webnn

Will not. This is the plan:

currently both onnxruntime-web and onnxruntime-web/webgpu supports WebNN. we keep both for testing and validation purpose.

once we think the time is mature, we will use WebGPU EP in onnxruntime-web and onnxruntime-web/webgpu will be deprecated. eventually everyone should use onnxruntime-web (WebGPU EP + WebNN EP) and there is going to be no JSEP.

fs-eire avatar Jun 18 '25 22:06 fs-eire

@Honry could you please help to test/validate if this change breaks WebNN? This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

@fs-eire, both work! BTW, will we have a build specific for WebNN? e.g. ort.webnn.min.js, onnxruntime-web/webnn

Will not. This is the plan:

currently both onnxruntime-web and onnxruntime-web/webgpu supports WebNN. we keep both for testing and validation purpose.

once we think the time is mature, we will use WebGPU EP in onnxruntime-web and onnxruntime-web/webgpu will be deprecated. eventually everyone should use onnxruntime-web (WebGPU EP + WebNN EP) and there is going to be no JSEP.

That make sense, thank you for the explanation!

Honry avatar Jun 18 '25 23:06 Honry