ShijieZZZZ

Hi @root20, I could not reproduce this.  What is your ds_report? Mine is ``` deepspeed info ................... 0.8.1+4af1f76a, 4af1f76a, master torch cuda version ............... 11.6 torch hip version ...................

Could you try ```import deepspeed.utils.zero_to_fp32``` to see if zero_to_fp32 can be used in that way.

Hi @achicu, I could not reproduce this issue with the [example you provided](https://gist.github.com/achicu/884e18d6ec599d1d2db574dd056a16ad) (with either self.param1 = torch.nn.Parameter(torch.ones(1)) or self.param1 = torch.nn.Parameter(torch.ones(2), both reaching the passing point). Could you please...

@achicu, feel free to re-open if you're still seeing issues here.