list_physical_devices("GPU")?

[CoryGH]

Is there a way to list GPUs available to tensorflow from node.js similar to how the python library can? Similarly, is it possible to direct specific operations to specific GPUs from within node.js via tfjs-node-gpu?

[gaikwadrahul8]

Hi, @CoryGH

Apologize for the delayed response and as far I know we are importing @tensorflow/tfjs-node-gpu and the module is accelerated by the TensorFlow C binary. But the GPU package runs tensor operations on the GPU with CUDA, so it's only available on Linux. This binding can be at least an order of magnitude faster than the other binding options.

This package currently only works with CUDA. You need to have CUDA installed on your machine with an NVIDIA graphics card before using tfjs-node-gpu.

I completely understood you're looking with tf.device('/device:gpu:1') or with tf.device('/device:gpu:0') in Node.js so you can run specific operations on specified devices like CPU or on different GPU's and it seems like at the moment we don't have such direct API like python(with tf.device) in TFJS and we have tf.setBackend which Sets the backend (cpu, webgl, wasm, etc) responsible for creating tensors and executing operations on those tensors. Returns a promise that resolves to a boolean if the backend initialization was successful. Please refer this comment also

If have I missed something here please let me know ? Thank you!

[CoryGH]

Is it possible to target an operation at a specific card if you have multiple GPUs installed, or does it load-balance between them?

[gaikwadrahul8]

@CoryGH

I found similar issues #1277 and #1551 and at the moment I'm not sure whether we can target specific operation at specific GPU card but as per those similar issues users observed memory allocation happened between 02 GPU's and please make sure you follow our official documentation for installation with CUDA® Toolkit 11.2 and cuDNN SDK 8.1.0. Thank you!

CC :@Linchenn

[gaikwadrahul8]

Hi, @CoryGH

I apologize for the delayed response and as far I know TensorFlow.js in Node.js offers the @tensorflow/tfjs-node-gpu package for GPU acceleration but it currently doesn't support explicitly targeting individual GPUs within your code. You cannot directly assign specific operations to particular GPUs.

The framework automatically implements load balancing across all available GPUs during training or inference. This approach simplifies the development process and ensures efficient resource utilization.

If I have missed something here please let me know ? Thank you for your understanding and patience.