PASSL icon indicating copy to clipboard operation
PASSL copied to clipboard

Published 20 hours ago • verified publisher icon PaddlePaddle

BYOL的预训练中好像使用了gt_label？

Open youqingxiaozhua opened this issue 2 years ago • 1 comments

  • 在byol的config 中设置了 num_classes=1000: https://github.com/PaddlePaddle/PASSL/blob/9d7a9fd4af41772e29120553dddab1c162e4cb70/configs/byol/byol_r50_IM.yaml#L34
  • 在model中设置了self.classifier = nn.Linear(embedding_dim, num_classes)，并且forward中将classif_out和label一起传给了head

image

https://github.com/PaddlePaddle/PASSL/blob/9d7a9fd4af41772e29120553dddab1c162e4cb70/passl/modeling/architectures/BYOL.py#L263

  • 在L2 Head中将对比loss和有监督的CE loss加在了一起返回

image

https://github.com/PaddlePaddle/PASSL/blob/9d7a9fd4af41772e29120553dddab1c162e4cb70/passl/modeling/heads/l2_head.py#L43

youqingxiaozhua avatar Nov 14 '22 12:11 youqingxiaozhua

问题是否还存在，建议使用mae 可以先转到 https://github.com/PaddlePaddle/PLSC/tree/master/task/ssl/mae 来使用

jerrywgz avatar Feb 05 '24 13:02 jerrywgz