Purpose of learnable scale and bias in the cosine classifier

[leosampaio]

Hi! Thank you for making the code and models public, the paper is very eye-opening!

I was trying to do a reimplementation of your code within our framework and came across the learnable parameters for a scale and bias in the ProtoNet implementation. These are used to scale and shift cosine similarities between query samples and prototypes:

https://github.com/hushell/pmf_cvpr22/blob/720b76d213cfda6c88113550c6dd40106b751ad2/models/protonet.py#L11

https://github.com/hushell/pmf_cvpr22/blob/720b76d213cfda6c88113550c6dd40106b751ad2/models/protonet.py#L26

What is the importance of these new parameters? Is there a reference that supports the inclusion of them in ProtoNet?

[hushell]

Thank you for your interests and sorry for the late reply! scale_cls is actually the temperature scaling for softmax to control the entropy of the multinomial distribution. bias is not necessary as it shifts all class scores and then won't change the CE loss for a finite bias value. The code of ProtoNet classifier is inherited from Gidaris et al. CVPR 2018