bigscience icon indicating copy to clipboard operation
bigscience copied to clipboard

Published 20 hours ago • verified publisher icon bigscience-workshop

What is the number of epochs of the final training?

Open cmsflash opened this issue 3 years ago • 0 comments

The config file lists the sample count of the dataset as 220M and a global batch size of 2048, which equates to ~107K steps per epoch. The main README says the total number of training steps is 95K, which means epoch 1 is not finished. However, the training chronicles suggest more than one epochs of training.

What is the number of epoch for the final training and what am I missing?

cmsflash avatar Aug 12 '22 23:08 cmsflash