Yiran Ma

Any update now? 4.36.2 definitely have the same issue! Which is the latest version that does not have this annoying bug?

> Any update now? 4.36.2 definitely have the same issue! Which is the latest version that does not have this annoying bug? Latest V4.37.1 still has the same issue in...

I just found that setting `save_on_each_node=False` in TrainingArguments works. See [#28009](https://github.com/huggingface/transformers/pull/28009)

Any progress? I encountered the same bug when using zero2-offload, while zero3-offload works correctly.