Set gpu index doesn't work

[shencuifeng]

When I set device=torch.device("cuda:1") it still load the model to cuda:0, how to load the model to different gpus?

[lmxyy]

Are you using comfyui or diffusers? Could you also share the context?

[shencuifeng]

I am using diffusers, and the code is:

device="cuda:1"
device=torch.device(device)
transformer = NunchakuFluxTransformer2dModel.from_pretrained("mit-han-lab/svdq-int4-flux.1-dev", device=device)

but it still load the model to device 'cuda:0'

[lmxyy]

I see. Could you set the CUDA_VISIBLE_DEVICES first? We are fixing this.

[lmxyy]

Just a heads-up: this issue will be resolved in the next v0.2.0 release. I will let you know then.

[ita9naiwa]

@lmxyy any update?

[lmxyy]

Got fixed in main branch.

[lmxyy]

This issue is fixed in v0.3.0dev0.