Multi-GPU usage for inference

[paulovasconcellos-hotmart]

Hello, thank you very much for this amazing contribution.

I was wondering if it's possible to use multi-GPU for inference. I have 8xA100, but when I use it for inference, just one of them is used throughout the whole process. What can I do to use all my GPUs for inference?

---

你好，非常感谢你做出的这项了不起的贡献。

我想知道是否可以使用多GPU进行推理。我有8个A100显卡，但是在推理过程中，只使用其中一个。我要怎样做才能在推理中使用所有的GPU呢？