finetune-anything icon indicating copy to clipboard operation
finetune-anything copied to clipboard

Problems with lr scheduler

Open xdedss opened this issue 5 months ago • 2 comments

https://github.com/ziqi-jin/finetune-anything/blob/1e5640d7796b078248b1c24a18ca34c6d834f025/extend_sam/init.py#L34C5-L34C18

The default args are not passed into get_scheduler in train.py

        max_epoch=1,
        n_epochs_init=50,
        n_epochs_decay=50,

As a result, cosine and linear scheduler does not work properly. cosine will oscillate rapidly between 1 and 0 and linear will decrease the lr to negative values, causing the loss to explode

xdedss avatar Jan 31 '24 01:01 xdedss