How to load model in all gpus during generation?

[Hins]

trafficstars

I have completed fine-tuning in 8 A100 gpus, when loading the ft model by "model=model.to("cuda")" it appeared OOM. And I have set 'os.environ['CUDA_VISIBLE_DEVICES'] = "0,1,2,3,4,5,6,7"' during generation.