transformers.js icon indicating copy to clipboard operation
transformers.js copied to clipboard

[WIP] Deno browser WASM/WebGPU support

Open xenova opened this issue 5 months ago • 4 comments
trafficstars

An experimental branch/PR to test running Transformers.js in Deno's WASM and WebGPU runtimes. This is a bit hacky, but it works... at least for now!

You can test this out by building this PR, and hosting the js files locally.

import { pipeline } from "http://localhost:8080/dist/transformers.js";

const extractor = await pipeline(
    "feature-extraction", "Xenova/all-MiniLM-L6-v2",
    // { device: "webgpu" }, // <-- Enable WebGPU acceleration
);

// Compute sentence embeddings
const sentences = ["Hello world", "This is an example sentence"];
const output = await extractor(sentences, { pooling: "mean", normalize: true });
console.log(output.tolist());

NOTE: This is different to running with the native EP (#1306), which will enable WebGPU support in Deno, via ONNXRuntime's Native WebGPU EP implementation.

There are some bugs/limitations with the current runtime (based on wgpu), e.g.,

An uncaught WebGPU validation error was raised: 
Shader 'Gather' parsing error: the `f16` enable-extension is not yet supported
  ┌─ wgsl:1:8
  │
1 │ enable f16;
  │        ^^^ this enable-extension specifies standard functionality which is not yet implemented in Naga
  │
  = note: Let Naga maintainers know that you ran into this at <https://github.com/gfx-rs/wgpu/issues/4384>, so they can prioritize it!

: the `f16` enable-extension is not yet supported
An uncaught WebGPU validation error was raised: ShaderModule with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalid
An uncaught WebGPU validation error was raised: In a set_pipeline command: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalid
An uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalid

but WASM works well 👍

cc @crowlKats for viz

xenova avatar Jun 20 '25 03:06 xenova