llm-foundry icon indicating copy to clipboard operation
llm-foundry copied to clipboard

Conversion Sharded -> Monolithic checkpoint

Open pretidav opened this issue 1 year ago • 1 comments

I was wondering if there was a straightforward way to convert from sharded to monolithic checkpoint for a subsequent conversion to hf format (not a direct conversion sharded -> hf). I've read you can define a monolithic callback saver, however I would like to use some "off training" way, simply reading and writing the checkpoint in the now format.

Thanks for all the answers.

pretidav avatar May 17 '24 13:05 pretidav