wanted: lr_cyclic() and lr_cosine_annealing_warm_restarts()

[skeydan]

I think it would be very useful to have learning rate schedulers

• lr_cyclic() (https://arxiv.org/abs/1506.01186, Python source at https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#CyclicLR), and
• lr_cosine_annealing_warm_restarts() (https://arxiv.org/abs/1608.03983, Python source at https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#CosineAnnealingWarmRestarts)

Would anyone be interested in implementing these?