EDGE icon indicating copy to clipboard operation
EDGE copied to clipboard

What is the best batch size without considering GPU memory?

Open lzyplayer opened this issue 1 year ago • 0 comments

Truly a masterpiece! And thank you for your willingness to share your work. I have a few questions about batch_size and would like to ask the following. The paper mentions the use of 4 A100 graphics cards and batch size of 512. Is the model trained in this batch_size better? If I want to use a different batch_size, is there a recommended data set split ratio? What batch_size is the checkpoint.pt provided in GoogleDrive trained under?

lzyplayer avatar May 19 '23 03:05 lzyplayer