after log2feats function， why do you want to multiply it bitwise with pos embedding?

[Sharkchili-Z]

pos_logits = (log_feats * pos_embs).sum(dim=-1) neg_logits = (log_feats * neg_embs).sum(dim=-1) Ask the boss to explain the two steps and why to multiply by bits. After reading this paper, I didn't understand the meaning of these two steps, so I asked for an answer.

[pmixer]

It's for scoring, just one vector dot another to get a scalar, the scalar(logits) later got sent to softmax function to normalize to the range of [0, 1], 0 for negative, 1 for postive.

Pls revisit some classical work like FM by surfing on github if interested and needed.