Question regarding peak learning rate and minimum learning rate in pretraining stages of BLIP-2

[L-Hugh]

Hello, I'm currently working on replicating BLIP-2 and I have some questions regarding the learning rate settings in the two pretraining stages.

1. Is the peak learning rate set to 1e-4 in both pretraining stages?
2. In the paper, the minimum learning rate for the second stage is mentioned as 5e-5, but in the configuration file lavis/projects/blip2/train/pretrain_stage2.yaml, the minimum learning rate is set to 1e-5. Can you please clarify the reason for this discrepancy?
3. What is the minimum learning rate for the first pretraining stage?

I would appreciate your assistance in understanding these settings. Thank you for your time and effort in developing this great project!