Lai Wei

icon indicating a website link https://waltonfuture.github.io/

icon company Shanghai Jiao Tong University icon location Shanghai icon location I am currently a first-year CS PhD student at SJTU.

Results 11 issues of Lai Wei

How to use multiple GPUs for inference?

6 comment

I want to use multiple GPUs for inference, and I use device_map='auto' to load the model. However, I always met that problem: Expected all tensors to be on the same...