yolov9 icon indicating copy to clipboard operation
yolov9 copied to clipboard

Is incremental training possible?

Open shubzk opened this issue 10 months ago • 2 comments

I have a dataset with 5000 images containing 21 classes. However, I get new 500 images every 2 days with the same classes. Can I use the previous 'best.pt' weight file to only train on the new data and still retain all the information learned from previous trainings? NOTE: The classes remain the same.

shubzk avatar Apr 08 '24 16:04 shubzk

Check issue #323 (for resuming) Howewer, you can just put the weights from your last workout as base weights

Youho99 avatar Apr 09 '24 07:04 Youho99

If the images are significantly different from the original dataset, the training will update the all weights for the backbone layers to extract features, it's recommended to wait until you accumulate a week's worth of images and train everything at once. This is because every time you train the backbone, the new images influence the existing weights, potentially yielding poor results in each epoch until the optimal weights are found.

However, if the images don't represent significant changes in the classes, then you can train just the HEAD, which consists freeze backbone weights and training only final layers of the model. This process is quicker, allowing you to train daily.

levipereira avatar Apr 25 '24 02:04 levipereira