Test results are all 0 when using the author's checkpoint

[XuMengyaAmy]

1. I used the provided caption_model_47.pth from xlan experiment and the following commands to do the test. However, all test metrics are 0 when I used the def decode_beam which is inspired by meshed-memory-transformer. https://github.com/JDAI-CV/image-captioning/blob/master/models/att_basic_model.py https://github.com/JDAI-CV/image-captioning/blob/master/models/xtransformer.py

And it's the same case for the xlan+transformer test. I feel the decode_beam (from m2transformer) has problems. Do you have the same issue? Can you help to solve it? Thanks.

What I changed in the decode_beam(self, **kwargs) is adding .long() after some variable because of the error issue. 2) What's inside coco_train_cider.pkl? How to generate this file for custom dataset? 3) And I noticed that you used input_seq and target seq. From my understanding, both of them are ground truth caption which represented by word index, correct? The only difference is 0 and -1 at the end. Can you provide the preprocess code which convert the ground truth caption to word index?

Test Results: python3.8 main_test.py --folder experiments/xlan --resume 47 Called with args: Namespace(folder='experiments/xlan', resume=47) /media/mmlab/data_2/mengya/Code/ImageCaption/image-captioning/lib/config.py:381: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. yaml_cfg = edict(yaml.load(f)) loading annotations into memory... Done (t=0.03s) creating index... index created! 139it [00:11, 12.16it/s] Loading and preparing results... DONE (t=0.01s) creating index... index created! tokenization... PTBTokenizer tokenized 307085 tokens at 1576551.76 tokens per second. PTBTokenizer tokenized 4999 tokens at 130967.27 tokens per second. setting up scorers... computing Bleu score... {'testlen': 0, 'reflen': 42485, 'guess': [0, 0, 0, 0], 'correct': [0, 0, 0, 0]} ratio: 2.3537719195009454e-20 Bleu_1: 0.000 Bleu_2: 0.000 Bleu_3: 0.000 Bleu_4: 0.000 computing METEOR score... METEOR: 0.000 computing Rouge score... ROUGE_L: 0.000 computing CIDEr score... CIDEr: 0.000 computing SPICE score... Parsing reference captions Parsing test captions SPICE evaluation took: 3.045 s SPICE: 0.000

[YehLi]

1. Please try to change "selected_beam = selected_idx / candidate_logprob.shape[-1]" to "selected_beam = selected_idx // candidate_logprob.shape[-1]" for new pytorch version. The code can work correctly in my server after this change (torch version 1.8.1).

2. tools/cider_cache.py is the code for coco_train_cider.pkl

3. tools/create_labels.py is the code for input_seq and target seq

[alice-cool]

1. Please try to change "selected_beam = selected_idx / candidate_logprob.shape[-1]" to "selected_beam = selected_idx // candidate_logprob.shape[-1]" for new pytorch version. The code can work correctly in my server after this change (torch version 1.8.1).
2. tools/cider_cache.py is the code for coco_train_cider.pkl
3. tools/create_labels.py is the code for input_seq and target seq

I also use python2 with usual way ,for example. padding "=" to making it but it also didn't work.

[shamimsareem]

Transformer main_test result all are zero , please help: [INFO: 2024-03-16 19:09:05,645] {'Bleu_1': 0.0, 'Bleu_2': 0.0, 'Bleu_3': 0.0, 'Bleu_4': 0.0, 'METEOR': 0.0, 'ROUGE_L': 0.0, 'C