MilaNLProc
Add patience to reduce LR as CTM argument
- Contextualized Topic Models version: 2.5.0
- Python version: 3.9.16
- Operating System: Ubuntu 20.04.6 LTS
Description
Currently, ReduceLROnPlateau function for CTM has a high hard-coded value for patience. So, whenever you want the LR to decrease during training we need to set the EarlyStopping patience to some value higher than 10, otherwise the train stops before the LR is actually decreased.
This is impractical because, waiting 10 epochs for the LR to decrease is a lot.
Propose to fix it
class CTM:
# ...
def __init__(
self,
# ...
reduce_on_plateau=False,
lr_patience: int = 3
# ...
# init lr scheduler
if self.reduce_on_plateau:
self.scheduler = ReduceLROnPlateau(self.optimizer, patience=sefl.lr_patience)
Hello!
do you want to send a PR to add this?
On Mon, Aug 28, 2023 at 8:40 AM Diogo Rodrigues @.***> wrote:
- Contextualized Topic Models version: 2.5.0
- Python version: 3.9.16
- Operating System: Ubuntu 20.04.6 LTS
Description
Currently, ReduceLROnPlateau function for CTM has a high hard-coded value for patience. So, whenever you want the LR to decrease during training we need to set the EarlyStopping patience to some value higher than 10, otherwise the train stops before the LR is actually decreased.
This is impractical because, waiting 10 epochs for the LR to decrease is a lot.
Code line https://github.com/MilaNLProc/contextualized-topic-models/blob/5dd3238dd3a49e8a792163b02d4c3f3b5deaae9f/contextualized_topic_models/models/ctm.py#L154C13-L154C76 Propose to fix it
class CTM:
# ... def __init__( self, # ... reduce_on_plateau=False, lr_patience: int = 3 # ...— Reply to this email directly, view it on GitHub https://github.com/MilaNLProc/contextualized-topic-models/issues/141, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARBSSZZGUSBZREYG5VLTY3XXQ4PJANCNFSM6AAAAAA4A57AEA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
