[Fix] schedulers with no max_steps param

[stevehuang52]

All LR schedulers in PyTorch do not have the max_steps parameter, so we should not add max_steps to their scheduler_args. Previous code tackle the problem in case-by-case manner, while here we try to solve the problem in a more general way.

To do so, we define a set of schedulers that don't have the max_steps parameter, and set the add_max_args_flag accordingly.

Similarly, we define a set of epoch-based schedulers to set the interval, while previous code only checks for ReduceLROnPlateau scheduler.

Besides, the previous code also has bugs, since it doesn't set the correct add_max_args_flag and interval when if 'args' in scheduler_config is True.

[titu1994]

@ericharper for approval

[evilc3]

Hey so with this pr the I don't need to define max_step param for any scheduler ?

[stevehuang52]

Hey so with this pr the I don't need to define max_step param for any scheduler ?

Not really, this PR aims to fix the bug that current code will add a max_steps param to some pytorch schedulers that don't require max_steps.

For those schedulers that do have a max_steps param, current NeMo code will calculate a max_steps if it's not explicitly specified by the user.

[github-actions[bot]]

This PR is stale because it has been open for 30 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.