Shenggui Li

> > > upgrade gcc version from 4.8.5 to 9.5.0 fix this issue, but met another issue " AttributeError : module 'math' has not attribute prod " > > >...

LGTM. Will merge when checks are passed.

Hi, seems some Github updates broke our CI checks. I have fixed the CI and can you rebase your code with the latest code so that the correct CI can...

How many steps have you trained?

> we are designing the new checkpoint io module to support checkpoint saving/loading from various formats, such as single model weights, huggingface style sharded weights and megatron-style sharded tensor weights,...

Another automation task that is required is doc test.

> Another automation task that is required is doc test. This should be part of the user experience as stated in #2579 .

@kurisusnowdeng is this PR still valid? If not, I will close it.

> Ok, I will convert this PR to draft for now as commits will trigger the CI repeatedly. @kurisusnowdeng do make it ready for review once it is ready :)

Hi, Colossal-AI is not compatible with torch 2.0 for now.