MTMT icon indicating copy to clipboard operation
MTMT copied to clipboard

why the MT work?

Open mountain-three opened this issue 2 years ago • 4 comments

in my opinion,the mt just make the teacher and student network's outputs as the same as possible,but the same result doesn't mean the result is a correct result,especially when the labeled data is smaller than unlable data

mountain-three avatar Jun 15 '22 11:06 mountain-three

With regard to the usefulness of unlabeled data, you can read this paper "A brief introduction to weakly supervised learning". Figure 3 in this paper maybe give you some enlightening.

eraserNut avatar Jun 16 '22 06:06 eraserNut

Thanks for reply,But in the paper, when you count the conherency loss,you use the same unlabel input to teacher and student network,and minimize the difference between the outputs.the same image into two similar model,the outputs ought to be the same.so I think your idea is a good way to let your model output variance low but not kind of semi-supervised

mountain-three avatar Jun 16 '22 07:06 mountain-three

A type of semi-supervised learning method is using consistency losses between multi-models. In this way, you can consider it a regularization method. For some evidence, you can read "Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results".

eraserNut avatar Jun 16 '22 07:06 eraserNut

Thank you very much!!

------------------ 原始邮件 ------------------ 发件人: "eraserNut/MTMT" @.>; 发送时间: 2022年6月16日(星期四) 下午3:55 @.>; @.@.>; 主题: Re: [eraserNut/MTMT] why the MT work? (Issue #36)

A type of semi-supervised learning method is using consistency losses between multi-models. In this way, you can consider it a regularization method. For some evidence, you can read "Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results".

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

mountain-three avatar Jun 19 '22 05:06 mountain-three