MART icon indicating copy to clipboard operation
MART copied to clipboard

Add support for `ChainedScheduler`s in `LitModular`.

Open dxoigmn opened this issue 1 year ago • 0 comments

Right now LitModular does not support ChainedScheduler. This is because a ChainedScheduler does not take an optimizer, but rather a list of schedulers: https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ChainedScheduler.html

LitModular assumes lr_scheduler is a callable that receives an Optimizer: https://github.com/IntelLabs/MART/blob/41ad18e3aa08b420267772bd197e8c610389d2cc/mart/models/modular.py#L122

My sense is the best way to fix this is to allow lr_scheduler to be a list and we can create the underlying ChainedScheduler within LitModular.

dxoigmn avatar Jul 10 '23 15:07 dxoigmn