Reproducing zero-shot temporal image classification results on the MS-CXR-T benchmark

[Wentap123]

I have made modifications to the code hi-ml-multimodal/test_multimodal/vlp/test_zero_shot_classification.py in order to replicate the zero-shot temporal image classification results on the MS-CXR-T benchmark. However, the performance is lower compared to the aforementioned results. The details are as follows:

While using the code, I attempted different seeds, but obtained the same result. This is because it solely utilizes the function "get_similarity_score_from_raw_data()" to derive a score, which differs from the section "F.4. Auto-regressive prompting for zero-shot temporal image classification" in the paper titled "Learning to Exploit Temporal Structure for Biomedical Vision–Language Processing."

Could you provide me with some insights regarding this matter?

[pepperbubble]

same question