Pipeline inference with gpu?

[kouohhashi]

Hi, I tried pipeline inference on Colab.

from pyannote.audio import Pipeline
pipeline = Pipeline.from_pretrained('pyannote/speaker-diarization')

It seems model was not on GPU.

!nvidia-smi

Fri Aug 19 01:27:06 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
| N/A   68C    P0    28W /  70W |   1300MiB / 15109MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Is there any option to do inference with GPU?

Thanks for advance.

[hbredin]

It does run on GPU by default when GPUs are available.

[kouohhashi]

Thank you fore responding me. So even if !nvidia-smi show no process , it's going to use gpu? It's a Colab environment so gpu is supposed to be accessible.

[hbredin]

Yes.

[GoldDRoge]

Can i control turn off or on gpu mode ??

[kouohhashi]

Sorry my bad. somehow newly created env did not detect gpu. I re-installed torch and gpu is detected now...