Tingchen Fu

I encounter a similar issue. I continually train a bloom-560m model and convert a saved checkpoint with the zero_to_fp32.py. But when I tried to reload the converted checkpoint without deepspeed:...

Thanks! @mayank31398 Sorry for the late response. I just tried your recipe and it works: ``` import torch from transformers import BloomForCausalLM,BloomConfig configuration = BloomConfig.from_pretrained('/apdcephfs/share_916081/tingchenfu/PLM/bloom-560m') model = BloomForCausalLM(configuration) reloaded =...