There is a issue about the order of "lr_scheduler.step()" and "optimizer.step()" in the training process.

[angleboy8]

UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.

The code of "main_train_psnr.py" is presented as follows:

-------------------------------

        **# 1) update learning rate**
        # -------------------------------
        model.update_learning_rate(current_step)

        # -------------------------------
        # 2) feed patch pairs
        # -------------------------------
        model.feed_data(train_data)

        # -------------------------------
        # 3) optimize parameters
        # -------------------------------
        model.optimize_parameters(current_step)

Since "optimizer.step()" should be placed in front of "lr_scheduler.step()" to obtain the right learning rate updates, the step (1) needs to be placed below the step (3) . The revised code is displayed as:

        # -------------------------------
        # 2) feed patch pairs
        # -------------------------------
        model.feed_data(train_data)

        # -------------------------------
        # 3) optimize parameters
        # -------------------------------
        model.optimize_parameters(current_step)

        **# 1) update learning rate**
        # -------------------------------
        model.update_learning_rate(current_step)