RuntimeError with BLOOMZ: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

[karim1104]

I'm facing the above error in both stage 1 and stage 2 when using BLOOMZ 3B and 560M. I tried adding "model.to(device)" and "model.to('cuda')" to main.py but neither worked. The error only appears when I switch from Llama to BLOOMZ.

[XuJing1022]

hi, whether you have figured out the reason?