grit

grit copied to clipboard

Hi, thank you for sharing this great work.

I’m trying to reproduce the paper result on the 5k Karpathy test split test set using the inference script, but I’m getting a lower scores:

Bleu_1: 0.810 Bleu_2: 0.655 Bleu_3: 0.510 Bleu_4: 0.388 METEOR: 0.295 ROUGE_L: 0.587 CIDEr: 1.333 SPICE: 0.230

And after some digging, the caption is not fully generated, I managed to duplicate the problem in Colab as well.

https://colab.research.google.com/drive/1BvtscubSujlxOFhOchVGNB79KkKYoMiH?usp=sharing

Thank you for reporting the issue. Have you figured it out? I will inspect the problem and inform you later.

Unfortunately, not yet, I have checked the config file and everything looks ok to me (beam search, max_length, ..etc). I have tested the code with only one GPU (A100) and (Colab GPU), and I'm not sure if this will affect the inference.

Thanks for further information. It doesn't affect the inference as I also used 1 GPU. I haven't figured out the issue. Let me check this in details.

Same issue here. There is something to do with not using model.eval() in the inference code. So the captions generated are not deterministic. Every time I run the inference_caption.py, it gives me different captions. However if I run the example caption generation code with model.eval(), it give me an incomplete caption "three sheep standing next to a".

Hello, Could anyone please help. Im using Colab I need to used part of the code (Swin transformer as backbone ) to extracted features!. I tried to run "extract_features.py" but Im facing the following errors, any help is apricated .

output 2024-01-09 23:36:18.083731: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-01-09 23:36:18.083802: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-01-09 23:36:18.085352: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-01-09 23:36:19.463628: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT