MUST icon indicating copy to clipboard operation
MUST copied to clipboard

Published 20 hours ago • verified publisher icon salesforce

Global-local Feature Alignment

Open 1024er opened this issue 3 years ago • 1 comments

image Hi, Have you tried the InfoNCE loss in Global-local Feature Alignment ?

[CLS] and [MSK] in the same sentence constitute positive pairs [CLS] and [MSK] in different sentence constitute negative pairs

1024er avatar Jul 09 '22 06:07 1024er

Hi, thanks for your question. We have tried InfoNCE, but it does not work as well as the squared distance. Our hypothesis is that InfoNCE is too "simple" and does not regularize the model well enough.

LiJunnan1992 avatar Jul 12 '22 07:07 LiJunnan1992