tez icon indicating copy to clipboard operation
tez copied to clipboard

Saving after training an epoch

Open 25icecreamflavors opened this issue 2 years ago • 3 comments

How to save the model after each epoch training? I use fit method for 5 epochs and do not really understand hot to save after each one. not only after the last one.

25icecreamflavors avatar Aug 17 '22 04:08 25icecreamflavors

you can use the EarlyStopping callback and save on each epoch where there is an improvement. here is an example: https://github.com/abhishekkrthakur/tez/blob/main/examples/image/digit_recognizer.py#L175

abhishekkrthakur avatar Aug 17 '22 07:08 abhishekkrthakur

I'm not sure, will this stop training if there is no improvement for next epoch (I need to continue training anyway)? And do this function saves a model after the whole epoch, not somewhere in the middle right?

25icecreamflavors avatar Aug 17 '22 18:08 25icecreamflavors

You can create a custom callback and use model.save() to save the model at each epoch or if you want to save some specific epochs, that can also be done and that would help in saving model after each epoch, You can use something similar to below code, it maynot perfect but you can create a custom callback similarly

`
class CustomSaver(tez.callbacks.Callback):

    def on_epoch_end(self, epoch, logs={}):

        if epoch <=20:  # or save after some epoch, upto 20

            self.model.save("model_{}.hd5".format(epoch))

`

gullayeshwantkumarruler avatar Apr 14 '23 13:04 gullayeshwantkumarruler