ITM Objectives task will not be enabled？

[CQUTWangHong]

hello, @dandelin

I have tracked the code at vilt_module.py

from the training_step function -> set_task function

pl_module.current_tasks = [
        k for k, v in pl_module.hparams.config["loss_names"].items() if v >= 1
    ]

ITM task will enabled only when v>=1

However, no matter what pre training task, all ITM parameters are set to 0.5 in config.py

Is there a problem with my understanding？

thank you!

[CQUTWangHong]

Another question

names = ["f30k_caption_karpathy_test"] when split == "val" or "test" at loading f30k dataset

but, it is necessary to run get_false_Image or get_false_text function during the test phase?

def get_suite(self, index):
        result = None
        while result is None:
            try:
                ret = dict()
                ret.update(self.get_image(index))
                if not self.image_only:
                    txt = self.get_text(index)
                    ret.update({"replica": True if txt["cap_index"] > 0 else False})
                    ret.update(txt)

                for i in range(self.draw_false_image):
——>                 ret.update(self.get_false_image(i))
                for i in range(self.draw_false_text):
——>                 ret.update(self.get_false_text(i))
                result = True
            except Exception as e:
                print(f"Error while read file idx {index} in {self.names[0]} -> {e}")
                index = random.randint(0, len(self.index_mapper) - 1)
        return ret

thx!

[CQUTWangHong]

hi, @dandelin

Run the rank loop before each epoch. It takes me a lot of time , so what the program rank loop is doing during fine-tune?

rank loop: 100%|████████████████████████████████████████████████████████████████████████| 250/250 [3:42:38<00:00, 53.43s/it]

thank you!