mmFormer icon indicating copy to clipboard operation
mmFormer copied to clipboard

Published 20 hours ago • verified publisher icon YaoZhang93

input for missing modality

Open ejawany95 opened this issue 1 year ago • 1 comments

Congratulations on your great work. However, I have a question why did you choose to apply delta for the missing modality in the middle of the network rather than at the input stage?

ejawany95 avatar Sep 20 '23 05:09 ejawany95

Thanks for your interest. During training the network, if we apply the delta at the input stage, it might disturb the training of the single-modal encoder.

YaoZhang93 avatar Sep 21 '23 05:09 YaoZhang93