self-critical.pytorch
self-critical.pytorch copied to clipboard
ruotianluo
Lower CIDEr values on CIDEr Optimization
I am training my captioning model for 32 epochs (with CE loss) and then doing SCST training. But my rewards increase in the negative and my CIDEr decreases. Any possible reasons for this? Thanks for the help in advance.
Could you provide the details of your scores, like CIDEr or Bleu and the model you used.
Are you using the same hyperparameters as in the readme? Can you check your learning rate?
Hi everyone. Thanks for your reply. @ruotianluo - I am using it for a little bit different setting but if I make the learning rate very small (5e-6), the cider starts to increase for a few epochs but then starts decreasing again. Do you suggest a much smaller learning rate? My cider before optimization is 114.5 and increases only upto 115.1.
@goelarushi can you tell me what is your setting? what feature, what model are you using?
@ruotianluo, I am using the bottom-up features and a model similar to the original BUTD paper.
@goelarushi if you run the script in README.md, is that doing the right thing?
Yes, when I try to run the same setting as yours it works. But when I try to take a part of your cider optimization to my code and setting, it behaves differently. my loss and net reward are -0.75 and -0.96 respectively (do not got a lot lower than this). Thanks for your help in advance.
It's hard for me to guess what is wrong if you are using your own code. Are you running on COCO?
I understand. I will try to debug and I hope i get close to the solution soon. I am running on COCO Karpathy splits.
Hi @fawazsammani . Thanks for your reply. I am indeed using similar code. Do let me know if you encounter similar problem while using CIDEr optim. I see my reward doesn't go down much after some epochs. Thanks.