pytorch-lightning icon indicating copy to clipboard operation
pytorch-lightning copied to clipboard

Modification of the current_epoch attribute or other interesting @properties without setters

Open rucky96 opened this issue 1 year ago • 0 comments

Description & Motivation

When saving a checkpoint during training, it would be valuable to have the ability to set the current_epoch attribute of the Trainer class. Currently, attempting to set current_epoch results in an AttributeError since it lacks a setter. The motivation for this feature is evident in scenarios where users want to resume training from a specific epoch, especially when working with LR Schedulers. For instance, if a checkpoint is saved at epoch 100, and a change in LR occurs at epoch 150, users may want to restart training from epoch 100 to observe the impact of the new LR. This feature would enhance flexibility in managing training progress and experimenting with different configurations.

Pitch

I propose adding a setter for the current_epoch attribute in the Trainer class. This would allow users to programmatically set the training epoch, facilitating the ability to resume training from a specific point. For example:

trainer = Trainer()
# ...
# After loading a checkpoint
checkpoint = torch.load('path/to/checkpoint.ckpt')
trainer.current_epoch = checkpoint['epoch']
# Now the trainer is set to resume training from the specified epoch

configurations without unnecessary training repetition.

Alternatives

An alternative approach would be to provide a method in the Trainer class explicitly designed for setting the current epoch. However, having a setter for current_epoch aligns with the intuitive way Python properties are typically used, offering a cleaner and more idiomatic solution.

Additional context

This feature becomes particularly relevant in situations where experimentation involves frequent changes in LR or other hyperparameters, allowing users to easily explore different configurations without unnecessary training repetition.

cc @borda

rucky96 avatar Oct 16 '23 17:10 rucky96