transformers.js icon indicating copy to clipboard operation
transformers.js copied to clipboard
verified publisher icon xenova

vulkan support for nodejs

Open ali-ghamdan opened this issue 8 months ago • 2 comments

Feature request

it looks like that transformers.js only can run on cuda if linux on nvidia and if there is an internal gpu, the cpu will only be used.

Motivation

Vulcan will allow to a lot of devices to leave the CPU speed, we all know that Vulkan will make it faster.

Your contribution

sorry but no.

ali-ghamdan avatar May 05 '25 18:05 ali-ghamdan

Hi there 👋 Vulkan support for Node.js may be available with the Native WebGPU EP, which will be released with Transformers.js soon (PR for tracking: https://github.com/huggingface/transformers.js/pull/1306).

cc @fs-eire to confirm?

xenova avatar May 05 '25 22:05 xenova

no. currently WebGPU EP is only enabled in Windows and macOS in Node.js binding. There are multiple reasons:

  • the current release pipeline does not yet support CUDA + WebGPU in the same build
  • WebGPU EP is not well tested under linux and/or vulkan. For example, f16 feature is disabled for all NVidia GPUs under vulkan, because of CTS tests are not passing.

WebGPU EP still works on linux with vulkan but the current status is it's not ready out of the box for linux. you can build it from source.

fs-eire avatar May 05 '25 23:05 fs-eire