Grid search YOLOX base_lr

[okotaku]

Describe the feature

mim grid search can search optimal lr like followings.

search_args = '--optim_wrapper.optimizer.lr 0.001 0.01 0.2'

But YOLOX lr search has a problem. We should search base_lr instead of optim_wrapper.optimizer.lr because of param_scheduler.　
https://github.com/open-mmlab/mmdetection/blob/dev-3.x/configs/yolox/yolox_s_8xb8-300e_coco.py#L175

Config is like following.

base_lr = 0.01
optim_wrapper = dict(
    type='OptimWrapper',
    optimizer=dict(
        type='SGD', lr=base_lr, momentum=0.9, weight_decay=5e-4,
        nesterov=True),
    paramwise_cfg=dict(norm_decay_mult=0., bias_decay_mult=0.))

param_scheduler = [
   ...
    dict(
        type='CosineAnnealingLR',
        eta_min=base_lr * 0.05,  # This part is a problem for lr grid search.
        begin=5,
        T_max=max_epochs - num_last_epochs,
        end=max_epochs - num_last_epochs,
        by_epoch=True,
        convert_to_iter_based=True),
    ...
]

However, we currently do not have the function to properly search lr and param_scheduler by searching base_lr.

Motivation

I request the following features.

1. mim gridsearch mmdet yolox_s_8xb8-300e_coco.py --work-dir tmp --gpus 1 --search-args '--base_lr 1e-2 1e-3'
2. Then, along with base_lr, the appropriate optim_wrapper.optimizer.lr and param_scheduler are set and the search can be performed.

[Chop1]

Yes, sometime the gridsearch require inter dependant values which is currently not supported. I would like this feature as well.

[ice-tong]

Hi @okotaku @Chop1 , thanks for the attention to mim gridsearch!

You are totally right that sometimes we need to search for an inter-dependant value that could affect multiple places in the configs.

Currently, I don't have a general solution to address this.

Let's leave this issue open, at some point we need to support it or someone has a good solution. ^_^