Olatunji Ruwase

> Another interesting experiment I conducted between regular Torch save+load vs. AIO save + torch load vs. safetensors save + load. Thanks for sharing these early results. But I am...

@jeromeku, you can start here: https://www.deepspeed.ai/tutorials/zeropp/

> Can this be accomplished given hpZ, and if so, what would be the appropriate config? No, this is not possible in hpZ.

@pacman100, thanks for asking. DeepSpeed has provided supported for multiple models since our release [DeepSpeed-Chat](https://github.com/microsoft/DeepSpeed/blob/master/blogs/deepspeed-chat/README.md) release in April 2023. DeepSpeed-Chat implementation is available [here](https://github.com/microsoft/DeepSpeedExamples/tree/master/applications/DeepSpeed-Chat). Here is a good entry point...

> Works as expected. I have one question I'd like to confirm. Do we need to save the status of data loader to avoid reusing data samples? This is a...