bge small 继续预训练loss

[parkLGW]

用4GB数据继续预训练，loss从14降到7，算正常的吗？有什么方法可以提升呢

[staoxiao]

实验中我们一般会降到4-5左右，可以再多训练一下

[parkLGW]

实验中我们一般会降到4-5左右，可以再多训练一下

好的，我再加一个epoch试试，感谢答复

[YanchengWang]

实验中我们一般会降到4-5左右，可以再多训练一下

Do you have any suggestions on the number of epochs in training bge-large-en? We are using a 200 GB dataset. And is there an expected training loss in beg-large pre-training?

Thanks!

[staoxiao]

实验中我们一般会降到4-5左右，可以再多训练一下

好的，我再加一个epoch试试，感谢答复

抱歉，弄混了，我们small模型的loss到6左右，所以到7也还行。large才是4左右。

[staoxiao]

实验中我们一般会降到4-5左右，可以再多训练一下

Do you have any suggestions on the number of epochs in training bge-large-en? We are using a 200 GB dataset. And is there an expected training loss in beg-large pre-training?

Thanks!

Roughly 1 to 2 epochs should suffice, with the loss around 5