Pretraining code

[JunzheJosephZhu]

Hi, How are the models pretrained? I notice that custom architectures like injector and extractors require rewriting the model, so I'm assuming you pretrained the model yourself? If that's correct, could you provide a link to the code for pretraining these models?

[duanduanduanyuchen]

Hi, How are the models pretrained? I notice that custom architectures like injector and extractors require rewriting the model, so I'm assuming you pretrained the model yourself? If that's correct, could you provide a link to the code for pretraining these models?

Hi! Our model only loads the pre-trained weights of the backbone(like ViT or BeiT). Other parts are inited randomly. So we can load the weights directly from the released weights without pre-train the model by ourselves.