FlagEmbedding icon indicating copy to clipboard operation
FlagEmbedding copied to clipboard

Published 20 hours ago • verified publisher icon FlagOpen

关于batch_size的计算方式

Open zhaobinNF opened this issue 1 year ago • 2 comments

您好，论文里提到的batch_size=19200，指的是Device_num X per_device_batch_size X accumulation_steps这样计算出来的结果吗

zhaobinNF avatar Jan 26 '24 08:01 zhaobinNF

您好，是Device_num X per_device_batch_size，我们没有开启accumulation_steps。

staoxiao avatar Jan 26 '24 09:01 staoxiao

您好，多谢您的回复，还想再问一下stage1训练时train_group_size您是设置成多少的

zhaobinNF avatar Jan 26 '24 09:01 zhaobinNF