transformers.js
transformers.js copied to clipboard
[WIP] Deno browser WASM/WebGPU support
trafficstars
An experimental branch/PR to test running Transformers.js in Deno's WASM and WebGPU runtimes. This is a bit hacky, but it works... at least for now!
You can test this out by building this PR, and hosting the js files locally.
import { pipeline } from "http://localhost:8080/dist/transformers.js";
const extractor = await pipeline(
"feature-extraction", "Xenova/all-MiniLM-L6-v2",
// { device: "webgpu" }, // <-- Enable WebGPU acceleration
);
// Compute sentence embeddings
const sentences = ["Hello world", "This is an example sentence"];
const output = await extractor(sentences, { pooling: "mean", normalize: true });
console.log(output.tolist());
NOTE: This is different to running with the native EP (#1306), which will enable WebGPU support in Deno, via ONNXRuntime's Native WebGPU EP implementation.
There are some bugs/limitations with the current runtime (based on wgpu), e.g.,
An uncaught WebGPU validation error was raised:
Shader 'Gather' parsing error: the `f16` enable-extension is not yet supported
┌─ wgsl:1:8
│
1 │ enable f16;
│ ^^^ this enable-extension specifies standard functionality which is not yet implemented in Naga
│
= note: Let Naga maintainers know that you ran into this at <https://github.com/gfx-rs/wgpu/issues/4384>, so they can prioritize it!
: the `f16` enable-extension is not yet supported
An uncaught WebGPU validation error was raised: ShaderModule with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalid
An uncaught WebGPU validation error was raised: In a set_pipeline command: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalid
An uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalid
but WASM works well 👍
cc @crowlKats for viz