Can't evaluate BLEU/METEOR/CIDEr scores while running eval.lua

[y657250418]

When I used the command:

th eval.lua -model /home/yh/checkpoint/model_id1-501-1448236541.t7 -image_folder /home/yh/mscoco/test2015 -num_images -1 -language_eval 1

I get the next error message:

loading annotations into memory... 0:00:00.634204 creating index... index created! using 0/81434 predictions Loading and preparing results...
Traceback (most recent call last): File "myeval.py", line 29, in cocoRes = coco.loadRes(resFile) File "/mnt/disk1/dandan1/yanghuan/neuraltalk2/neuraltalk2-master/coco-caption/pycocotools/coco.py", line 280, in loadRes if 'caption' in anns[0]: IndexError: list index out of range /home/wonglab/torch/install/bin/luajit: ./misc/utils.lua:17: attempt to index local 'file' (a nil value) stack traceback: ./misc/utils.lua:17: in function 'read_json' ./misc/net_utils.lua:202: in function 'language_eval' eval.lua:167: in function 'eval_split' eval.lua:173: in main chunk [C]: in function 'dofile' ...glab/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

Who can help me?

[xinghedyc]

I got the same error.

[y657250418]

@xinghedyc I find that whatever the image I try to evaluate, it will copy a new image and rename the image_id. So the image_id in the valevalscript.json created by the function net_utils.language_eval(predictions, id) is different from the id in the captions_val2014.json, the image can't get the ground truth. it always appear

creating index... index created! using # 0/40504 predictions

When I use the command which not include the -image_folder parameter, I get the model scores.

[Afeihan]

@y657250418 I got the same error,can you give me a specific example how to run it ?? when I use the command which not include the -image_folder parameter as follow:

$ th eval.lua -model model_id1-501-1448236541.t7 -language_eval 1

I failed .

[y657250418]

@Afeihan th eval.lua -model /home/.../model200000.t7 -language_eval 1 -num_images 1000

[HuadongTang]

@y657250418 I got the same error when I evaluate on CIDEr/Bleu... I am from China and I think you are also from China . Can I have your qq number so that I can ask something about the neuraltalk2 ? 多谢 . My qq number is 852077607

[mymuli]

我知道怎么解决了.... 在eval阶段，设置命令-language_eval 0....如果-language_eval 1, 会报错： if 'caption' in anns[0]: IndexError: list index out of range