philippmtk

Load Bloom Optimizer State (i.e. Bloom 1B1)

2

Hi, I want to continue training of the Bloom model. To start simple, I want to load the 1.1B model into the BigScience Megatron-DeepSpeed library. I tried to run pretrain_gpt.py...