ColossalAI-Examples icon indicating copy to clipboard operation
ColossalAI-Examples copied to clipboard

[bert] update the zero pretraining and finetuning with new zero

Open FrankLeeeee opened this issue 3 years ago • 0 comments
trafficstars

Fixed the training script such that len(dataloader) works fine. This script is updated with the new zero api as well.

FrankLeeeee avatar Jul 27 '22 10:07 FrankLeeeee